apache 2.4 tuning and Keepalive

https://servercheck.in/blog/3-small-tweaks-make-apache-fly

Note: There is a good discussion on Reddit about situations when KeepAlive may be helpful and may, in fact, help Apache (or your server in general) perform better, like if you have an AJAX-heavy site, or if you are serving many requests to mobile or international users. Additionally, if you use a reverse proxy like squid, Varnish, or Nginx in front of Apache, KeepAlive doesn't have the same cost in terms of memory and process usage.

KeepAlive does one simple thing: destroy Apache's ability to handle many concurrent requests. Well, that and speed up existing connections by allowing them to download all assets before closing a TCP connection between a browser and your server (see note above).

This feature was designed to help ensure a browser could load the HTML, some images, stylesheets, etc. from your server all within one connection. Before people started using CDNs like CloudFlare or Amazon S3 for static content, and when most people had internet connections with hundreds of milliseconds of latency, this setting was much more valuable. (It still could be, in some circumstances, for mobile clients on 3G or LTE networks, or if you have an AJAX-heavy site.)

However, for many websites today, either setting the KeepAliveTimeout to a lower value like 1-5 seconds or switching KeepAlive off altogether will be a much better option.

Real-world example: When I launched Server Check.in with this thread on Hacker News, the post made the HN front page, meaning I was getting upwards of 10-20 requests per second for an hour or so. I noticed quickly that the server's load was under 1.00, and nothing seemed awry, but trying to access https://servercheck.in/ was taking 30 seconds or longer!

Turning off KeepAlive allowed the server to serve more users more quickly, since I didn't have a bunch of httpd threads sitting connected to browsers that had already received all the content they needed. If I want to turn it back on at some point, I'll make sure to set it lower than the default of 30 seconds!

Caveat: Now, in my case, I have the MaxClients directive set to 45, because with Drupal, I've made sure I don't spawn more threads than my server's limited memory can handle (I'll get to the why's for this below). If you are serving only static/cached content, and don't need to have a bunch of memory-hogging httpd threads for a PHP application like Drupal, you may be able to live with KeepAlive (make sure the timeout is low, still!), and a much larger MaxClients setting.

MaxClients

Queuing requests may save your server from swapping.

This setting helps Apache fly when your server is getting hit hard, say by a post from Reddit, Hacker News, or some fast and furious marketing campaign.

MaxClients is pretty self-descriptive: it's a setting that tells Apache the maximum number of clients it should serve simultaneously. It's important that you choose a sane value for this setting to prevent any number of bad things from happening:

• Set it too low and you might cause people to wait for Apache to respond to their request while your server is almost sitting idle, with plenty of RAM and CPU to spare.
• Set it too high and watch your server die in a slow, painful death as Apache runs out of RAM and starts swapping to disk. And watch page response times go from milliseconds to seconds or even minutes.

There's a really simple way you can know exactly how high to set MaxClients:

(Total RAM - RAM used for Linux, MySQL, etc.) / Average httpd process size.

To get the average httpd process size, log into your server and run the following on the command line: ps aux | grep 'httpd' | awk '{print $6/1024 " MB";}'. This will output a list of process sizes. If you want to calculate the average on-the-fly, try the command ps aux | grep 'httpd' | awk '{print $6/1024;}' | awk '{avg += ($1 - avg) / NR;} END {print avg " MB";}'. It's best to run this a few different times throughout a normal day to see if there's any major fluctuation in process size.

Let's say your average process is 50 MB (fairly common if you're using PHP and a CMS like Wordpress or Drupal!). You have a VPS with 2 GB of RAM, and you have about .5 GB allocated to MySQL (you can find the full amount with the command ps aux | grep 'mysql' | awk '{print $6/1024 " MB";}'). Let's leave a little overhead for Linux and other bits on the server, say .4 GB:

(2000 MB - 900 MB) / 50 MB = 22

You should set the MaxClients directive to 22, maybe even a little lower to allow for more overhead when your server is getting hammered. Apache's default is usually > 200—this means if your server gets hit by a more than 22 people in a short period of time, it will slow to a crawl, since Apache will gobble up all the RAM!

Setting the MaxClients value lower will allow Apache to simply queue up further requests until it gets a free thread (remember, also, that if you have KeepAlive on, threads will be locked up for [KeepAliveTimeout] seconds!). This might seem like a bad thing... but it's a lot better doing this and serving queued requests quickly (since Apache is working from RAM and not swap space), than trying to serve a hundred requests at once and having massive slowdowns for everyone!

Real-world example: One site I work with had a page pick up tons of traffic over the course of a few hours, and the web server (a small 2 GB VPS running LAMP) was getting hammered by up to 100 separate requests per second. We had the MaxClients setting a bit too high, and the server immediately started swapping. It was hard even connecting to the server via SSH, and when I checked top, the server was using about 4 GB of swap (yikes!), and the CPU was spiking around 45 (normally around .2-.3). Once we set MaxClients to a sane amount and restarted Apache, the server was able to get most pages served in 2-3 seconds (instead of 2-3 minutes—ifever). Once traffic died down, we were serving pages in less than 1 second again.

AllowOverride

AllowOverride is basically Apache's way of allowing developers to be lazy. Instead of using a global configuration file and includes, Apache lets you stick an '.htaccess' file in any directory, anywhere, and every time anyone requests any page through Apache, Apache will recursively scan every directory from the site's root to apply rules in .htaccess files.

This is very convenient—especially in situations like shared hosting, where the hosting provider can't be bothered to include hundreds of extra configuration directives in the main httpd.conf file, or (for security purposes) allow everyone on the server to have their own configuration files included via httpd.conf!

However, if you are running only one or two websites on a server, or you generally have full control of the server, you can turn off this convenient feature and simply add the rules into httpd.conf or an include (to add an include, just use something like Include /home/user/public_html/.htaccess inside the VirtualHosts directive for your server, along with the rule AllowOverride None.

Real-world example: The impact of this change on server performance varies greatly, and depends mostly on how many directory-levels deep you have files that are served on a given page request. For most people, though, there may be a file in something like /home/user/public_html/sites/sitename/files/images/cached/large/thumbnail/image.jpg. In this case, Apache has to recurse through 10 levels of directories, parsing all found .htaccess files, just to serve this one image file. For one Drupal site, though, which is a VPS with a virtualized disk on a SAN, I've measured 7-10% faster individual page loads.

Caveat: Many CMSes like Wordpress and Drupal rely on .htaccess rules for certain functionality, like 'Clean URLs', www-to-non-www redirects, file upload protections, and static content caching. Make sure you include any relevant .htaccess files in your VirtualHosts directive if you set AllowOverrides to None. See more: How to make Apache faster for Drupal.

Also, the real-world speedup you'll get when using a server that has a dedicated, real hard drive or SSD will be much smaller than a server that has a slow drive or uses NAS/SAN storage.

Conclusion

These are just three of 'low-hanging fruit' performance-related settings buried in your Apache httpd.conf file; many others will have a big impact on your site's performance as well. With proper tuning, and an eye towards memory usage, you can make Apache as scalable, fast and lightweight as almost any web server.

The problem

By default apache2 is configured to support 150 concurrent connections. This forces all parallel requests beyond that limit to wait. Especially if, for example, active sync clients maintain a permanent connection for push events to arrive.

To calculate the number of concurrent sessions each of your apache nodes must be able to provide, consider the total number of concurrent sessions in your OX cluster and divide it by the number of apache nodes involved in serving those sessions. Then add some room for statistical fluctuations.

The solution

This is an example configuration to provide 8000 concurrent connections. Please ensure that your apache is using the mpm_worker module. This allows us to serve lots of concurrent connections by using less RAM than with mpm_prefork as we are going to start much less processes.

<IfModule mpm_worker_module>
    ServerLimit             250
    StartServers             10
    MinSpareThreads          75
    MaxSpareThreads         250 
    ThreadLimit              64
    ThreadsPerChild          32
    MaxClients             8000
    MaxRequestsPerChild   10000
</IfModule>

Short explanation of the parameters:

ServerLimit	Declares the maximum number of running apache processes. If you change this value you have to restart the daemon.
StartServers	The number of processes to start initially when starting the apache daemon.
MinSpareThreads/MaxSpareThreads	This regulates how many threads may stay idle without being killed. Apache regulates this on its own very well with default values.
ThreadsPerChild	How many threads can be created per process. Can be changed during a reload.
ThreadLimit	ThreadsPerChild can be configured as high as this value during runtime. If you change this value you have to restart the daemon.
MaxClients	This declares how many concurrent connections we provide. Devided by ThreadsPerChild you get the suitable ServerLimit value. May be less than ServerLimit * ThreadsPerChild to reserve some resources that can be engaged during runtime with increasing MaxClients and reloading the configuration.
MaxRequestsPerChild	Defines the number of Connections that a process can handle during its lifetime (keep-alives are counted once). After that it will be killed. This can be used to prevent possible apache memory leaks. If set to 0 the lifetime is infinite.