Archive for May, 2006

Apache2 Documentation

Apache 2 Errors

Got this error:


configuration error:  couldn't check access.  No groups file?: /awstats/awstats.pl

Many things cause this error, but in my case, mod_auth was not enabled. Duh.

Apache2 Performance

Update
We’ve switched to Prefork MPM for stability and resource management. Since this server is also a high volume email server and stability is the most important aspect to its operations, we are scaling back on performance issues to increase stability. The other services also affect the values we use to setup preform mpm:
StartServers 5 - not starting and stopping the daemon too often, so this is somewhat moot.

MinSpareServers 5 - ready for more action
MaxSpareServers 25 - keeping this smaller so that unused Apache resources can be lent to email services when needed.
MaxClients 256 - I don’t want to run out of RAM!
MaxRequestsPerChild - 250 - we have awesome processors so I don’t mind restarting processes.

I also turned off KeepAlive.

The site is static so I’m mostly concerned with reliability, robustness, and uptime, rather than worrying about performance flexibility.

Old setup

I run a high volume server, 1,000,000 hits / day. Thats hits, not unique visitors. Approximately 10,000 visitors / day. Our site is mostly static. First off we have a huge server with 2GB of ram, then a RAID array with a good controller card, then these settings for Apache2:

  • Use mpm_worker - this is a multi-threaded server, using less memory.
  • Divide the ram by 5 for the max number of clients
  • Turn down timeout to 30 seconds
  • KeepAlive On
  • Unlimited Keepalive requests - important!
  • KeepAliveTimeout 2 - important!
  • Our worker settings:

# worker MPM [DEFAULT IF USE=threads]
# StartServers: initial number of server processes to start
# MaxClients: maximum number of simultaneous client connections
# MinSpareThreads: minimum number of worker threads which are kept spare
# MaxSpareThreads: maximum number of worker threads which are kept spare
# ThreadsPerChild: constant number of worker threads in each server process
# MaxRequestsPerChild: maximum number of requests a server process serves
<IfModule worker.c>
    StartServers        25
    MaxClients         400
    MinSpareThreads     50 
    MaxSpareThreads     75
    ThreadsPerChild     25   
    MaxRequestsPerChild  0
</IfModule>

Mostly thanks to this post, and some trial and error:
http://virtualthreads.blogspot.com/2006/01/tuning-apache-part-1.html

Apache2 on Debian

Notes

  • a2ensite and a2dissite are automated ways of enable and dissabling virtual hosts for Apache2 on Debian. Strange to me, but seems like it might come in handy at some point.
  • To enable mod_auth_mysql, I a2enmod auth_mysql, but I’m getting an error:

Invalid command 'AuthMySQLEnable', perhaps mis-spelled or defined by a module not included in the server configuration
  • The directives for mod_auth_mysql are totally different on Debian:

http://polestar.org/auth_mysql/USAGE.html


Setting up OpenSSL on Debian for Apache2

  1. a2enmod ssl
  2. cd /etc/apache2/ssl
  3. cd /etc/ssl
  4. nano openssl.cnf
  5. edit your defaults to minimize typing later on
  6. openssl req -new > new.cert.csr
  7. openssl rsa -in privkey.pem -out new.cert.key
  8. openssl x509 -in new.cert.csr -out new.cert.cert -req -signkey new.cert.key -days 1095
  9. cp /usr/lib/ssl/misc/new.cert.cert /etc/apache2/ssl/
  10. cp /usr/lib/ssl/misc/new.cert.key /etc/apache2/ssl/

Notes

  • Debian complains if you mix non-port specified IP wildcards, so always use *:80, *:443, or whatever port you are using to specify NameVirtualHosts.

External Related Links:
http://www.gentoo.org/doc/en/virt-mail-howto.xml

Setting up mod_dav over SSL with MySQL authentication

  1. On gentoo, edited /etc/conf.d/apache2 to include dav and dav_fs
  2. On gentoo, emerge mod_auth_mysql
  3. Getting annoying error from apache: End of file found: SSL handshake interrupted by system [Hint: Stop button pressed in browser?!]
  4. Boy the apache configuration was a pain, but I finally got it working.

Notes

  • Mac OS X 10.3 doesn’t support https webdav. Though you can use Goliath.
  • mod_dav won’t work with symlinks. Don’t even try it.
  • User’s attached to a mod_dav file share access the share with the same userid that apache is running. This can be a security issue if you are publically serving the same files.

Questions

  • Is it possible to "approximate" a virtual file system with
    • additional metadata from xml file
    • dynamic (possible?) Apache aliases and mod_rewrites - probably no easy way to automate this without automatically rewriting .htaccess files
    • user authentication and name service switch from a flexible data structure

External Links:
Mod Auth Mysql

Web DAV Windows XP


Apache is awesome web server software.

mod_dav

Mod_dav provides file sharing not unlike Samba or AFP.

mod_auth_mysql
Mod_auth_mysql is an Apache module which provides for authentication via MySQL.

The combination of Apache, mod_dav, and ssl is an incredible alternative and in my humble opinion improvement to a setup with a VPN (openVPN, PPTP, or L2TP/IPSec) with a Samba or AFP share.

External Links

Debian Power Management

See what capabilities are available:


box:/# cat /proc/acpi/processor/CPU0/power
active state:            C2
max_cstate:              C8
bus master activity:     00000000
states:
    C1:                  type[C1] promotion[C2] demotion[--] latency[000] usage[00002220]
   *C2:                  type[C2] promotion[--] demotion[C1] latency[090] usage[00152026]


ps2pdf Notes

Good overview page: http://www.cs.wisc.edu/~ghost/doc/cvs/Ps2pdf.htm

CMYK: ps2pdf -dProcessColorModel=/DeviceCMYK

There are some presets to consider which adjust the right amount of compression for various mediums:

-dCompressFonts=boolean
Defines whether ps2pdf will compress embedded fonts in the output. The default value is true; the false setting is intended only for debugging.

-dPDFSETTINGS=configuration
Presets the "distiller parameters" to one of four predefined settings:

  • /screen selects low-resolution output similar to the Acrobat Distiller "Screen Optimized" setting.
  • /ebook selects medium-resolution output similar to the Acrobat Distiller "eBook" setting.
  • /printer selects output similar to the Acrobat Distiller "Print Optimized" setting.
  • /prepress selects output similar to Acrobat Distiller "Prepress Optimized" setting.
  • /default selects output intended to be useful across a wide variety of uses, possibly at the expense of a larger output file.

The best way to make a low file sized PDF is to print to postscript in Illustrator, then use this command to convert to PDF:


ps2pdf -dPDFSETTINGS=/screen -dProcessColorModel=/DeviceCMYK filename.ai.ps

Even more interesting, the quality (fonts, etc) of the Postscript file will depend on the Postscript driver you use! Use the Apple Color Laser driver.

HTMLDOC


/usr/local/bin/htmldoc --book --no-links --no-title --compression=8 --gray --header "..." --no-toc -v -f _final_output.pdf *.html

/usr/local/bin/htmldoc --book --no-links --no-title --tocheader "..." --toclevels 1 --compression=8 --gray --header "..." --toctitle "ALL Journal" -v -f _final_output.pdf *.html

Why we choose Debian

We chose debian for its reliability. They also have a logical social contract:

http://www.debian.org/social_contract

The social contract is a promise from Debian to it’s users. Feel free to read through it.

The apt tool was originally Debian exclusive. However, many different distros now use this tool. Debian’s repositories are still by far the best out there. They contain the most packages, and it works 99% of the time. (Regardless of stable/testing/unstable)

UPDATE November 24, 2007: Personally, I stick with testing or stable for the most part.



Freeradius Documentation

emerge freeradius failed on gentoo:
>>> Test phase [not enabled]: net-dialup/freeradius-1.1.1-r1

>>> Install freeradius-1.1.1-r1 into /var/tmp/portage/freeradius-1.1.1-r1/image/ category net-dialup
* ‘enewgroup()’ called from ‘install()’ which is not a pkg_* function.
* Package fails at QA and at life. Please file a bug.

!!! ERROR: net-dialup/freeradius-1.1.1-r1 failed.
!!! Function enewgroup, Line 648, Exitcode 0
!!! Bad package! enewgroup is only for use in pkg_* functions!
!!! If you need support, post the topmost build error, NOT this status message.

traffic control tc Documentation

Bandwidth Throttling on Linux

To setup bandwidth throttling on linux, you can use advanced linux routing techniques. Install a package called iproute2, and it should include the "tc" binary command.

tc - traffic control

Traffic control on linux is done using the tc command. It can setup kernel packet queuing management disciplies, and you have incredible control over how the packets are released to the network. You can also have limited control over incoming packets using an ingress queue.

My tc notes

TBF is causing rsync and scp to report back very low transfer rates. Something with this is not right because Fugu, an SCP client reports accurate transfer rates.

External tc - related links
http://snafu.freedom.org/linux2.2/iproute-notes.html

http://www.opalsoft.net/qos/