Posts

Apache 2.4 Upgrade and the “Invalid Command ‘Order'” Error

The new Apache 2.4 has been released a few weeks ago and I decided to use this version while installing a new server (I compiled it from scratch rather that using an rpm or deb).

After using one of my tried and true Apache configuration files, I received this error on start:

Starting httpd: AH00526: Syntax error on line 104 of /usr/local/apache2/conf/httpd.conf:
Invalid command 'Order', perhaps misspelled or defined by a module not included in the server configuration

Common wisdom would imply that I should make sure the authz_host module is installed (LoadModule authz_host_module modules/mod_authz_host.so), however this just was not working.

Finally, I discovered that the Order command has been removed from Apache 2.4! According to the upgrade notes for Apache 2.4:

In 2.2, access control based on client hostname, IP address, and other characteristics of client requests was done using the directives Order, Allow, Deny, and Satisfy.

In 2.4, such access control is done in the same way as other authorization checks, using the new module mod_authz_host. The old access control idioms should be replaced by the new authentication mechanisms, although for compatibility with old configurations, the new module mod_access_compat is provided.

Basically, the Order command is depreciated.

In my case, I replaced the lines:

Order deny,allow
Deny from all

with:

Require all denied

Also make sure both of these modules are loaded:

LoadModule authz_core_module modules/mod_authz_core.so
LoadModule authz_host_module modules/mod_authz_host.so

Easy enough, but just be aware that there are several configuration changes between 2.2 and 2.4 which render your old Apache configuration files unusable.

How to Stop an Apache DDoS Attack with mod_evasive

The first inkling that I had a problem with a DDoS (Distributed Denial of Service) attack was a note sent to my inbox:

lfd on server1.myhostname.com: High 5 minute load average alert – 89.14

Apache DDoS

My initial thought was that a site on my server was getting Slashdotted or encountering the Digg or Reddit effect. I run Chartbeat on several sites where this occasionally happens and I will usually get an alert from them first. A quick look at the Extended status page from Apache showed that I had a much different kind of problem.

If a site is getting a lot of “good” natural traffic you will see a few things:

  • clients will be requesting all kinds of files from your site as a normal web browser would, and
  • the referring agent will show the link of the sending site, which you can verify.

In my case I had about 400 or so IP addresses requesting “/” from a little trafficked site of mine. Fortunately my Apache is well-tuned enough that it did not take the server down, which was crucial for diagnosing this problem. Otherwise the entire server may down and your only option is to reboot and stop apache from starting on boot. My apache logs showed the requests:

186.58.179.33 - - [21/Oct/2010:00:10:06 -0400] "GET / HTTP/1.1" 200 12474 "-" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.0; WOW64; Trident/4.0; SLCC1; .NET CLR 2.0.50727; .NET CLR 3.5.21022; .NET
CLR 3.5.30729; .NET CLR 3.0.30618)"
189.76.197.117 - - [21/Oct/2010:00:10:06 -0400] "GET / HTTP/1.1" 200 12474 "-" "Mozilla/5.0 (Windows; U; Windows NT 5.1; ru; rv:1.8.1.19) Gecko/20081201 Firefox/2.0.0.19"
186.58.179.33 - - [21/Oct/2010:00:10:06 -0400] "GET / HTTP/1.1" 200 12474 "-" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.0; WOW64; Trident/4.0; SLCC1; .NET CLR 2.0.50727; .NET CLR 3.5.21022; .NET
CLR 3.5.30729; .NET CLR 3.0.30618)"
186.6.168.11 - - [21/Oct/2010:00:10:07 -0400] "GET / HTTP/1.1" 200 12474 "-" "Mozilla/4.0 (compatible; MSIE 5.0; Windows 2000) Opera 6.03 [en]"
197.0.165.121 - - [21/Oct/2010:00:10:07 -0400] "GET / HTTP/1.1" 200 12474 "-" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/525.19 (KHTML, like Gecko) Chrome/0.4.154.25 Safari/525.19"
189.76.197.117 - - [21/Oct/2010:00:10:07 -0400] "GET / HTTP/1.1" 200 12474 "-" "Mozilla/5.0 (Windows; U; Windows NT 5.1; ru; rv:1.8.1.19) Gecko/20081201 Firefox/2.0.0.19"
197.0.165.121 - - [21/Oct/2010:00:10:07 -0400] "GET / HTTP/1.1" 200 12474 "-" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/525.19 (KHTML, like Gecko) Chrome/0.4.154.25 Safari/525.19"
186.6.168.11 - - [21/Oct/2010:00:10:07 -0400] "GET / HTTP/1.1" 200 12474 "-" "Mozilla/4.0 (compatible; MSIE 5.0; Windows 2000) Opera 6.03 [en]"

I found out this is from a type of DoS called a sloworis attack. In this attack, a HTTP request is made to the server and the connection is ‘held open’, making multiple requests. A compromised botnet is used to hammer the server from all over the world on many different connections and IP addresses. From the standpoint of Apache, these are legitimate connections despite the frequency of them. It simulates someone sitting at a browser and hitting the refresh command a few times a second. While not a devastating attack with regards to bandwidth, it ties up your server and rejects legitimate connections.

Fortunately after some asking around I found the mod_evasive Apache module. This module is a very basic one that has a simple function: it will keep a hash table of IPs and pages requested and when a threshold is reached on a page or site it will “block” the IP with a 403 “Forbidden” error.

Installing the module is easy:

  1. Download the module onto your server:

    # wget http://www.zdziarski.com/blog/wp-content/uploads/2010/02/mod_evasive_1.10.1.tar.gz

  2. Run the apache apxs command on the module which will compile it install it into your httpd.conf file (for Apache 2.0 – you are running 2.0 aren’t you?):

    # apxs -cia mod_evasive20.c

  3. Set up the configuration file. You can enter the following into your httpd.conf main server configuration (for Apache 2.0 again)


    DOSHashTableSize 3097
    DOSPageCount 3
    DOSSiteCount 100
    DOSPageInterval 3
    DOSSiteInterval 5
    DOSBlockingPeriod 300
    DOSLogDir "/var/log/httpd/modevasive/"
    DOSEmailNotify your@emailaddress.com

Now this works great to throw a forbidden page to the client, but that client is still taking up a slot on your server. Once all of your slots are filled up, other requests are queue and therefore your server is lethargic to non-responsive to real requests.

To fix this problem the mod_evasive module has a DOSSystemCommand option. Using this option you can have your server execute a command when a client trips the mod_evasive triggers. I use ConfigServerFirewall (csf) on my server, so I added the following command which specifies a 1 hour firewall ban on the IP, which effectively drops all of its traffic to the server:

DOSSystemCommand "/usr/bin/sudo /usr/sbin/csf -td %s 3600"

But wait a second! Apache doesn’t have access to the firewall normally. This is my one reservation about this proceedure. You need to give Apache access to the firewall programs (as root) via sudo so that it can execute this firewall block. This has other security implications, especially if you are on a multitenant server. We use visudo to do this.

# visudo

Add the following to the file:

User_Alias APACHE = apache
Cmnd_Alias FIREWALL = /sbin/iptables, /usr/sbin/csf, /sbin/ifconfig, /sbin/route
APACHE ALL = (ALL) NOPASSWD: FIREWALL

Where apache is the apache user (typically web, www, apache or httpd – this depends on the system) and the FIREWALL binaries are the ones used in the csf script.

Now my system watches for 3 or more connections to the same page in 3 seconds and not only serves a forbidden response to them but will drop their traffic completely via iptables. Within about 30 seconds my server load was back to normal, serving connections faithfully just like any other time.

This module worked great in my situation, but DDoS attacks come in many flavors and sizes. You may have luck with the mod_qos Apache module, where you can fine tune connections to certain pages. If an attack is bad enough, it is possible you will need to move to a hardware based solution because you can only do so much at the server level. A hardware firewall mixed with load balancers, caches like varnish, and other tricks can help to mitigate these DoS attacks.

Password Protected Folder Gives 404 Not Found in WordPress Installation Sub-folder.

Wordpress

Came across this little maddening issue again today after fixing it a few months back. I created a directory that is password protected using a .htaccess file. However, when trying to access this folder or anything under this directory, a File Not Found 404 error from WordPress is displayed before it even prompts you for the password. The problem here lays within the main WordPress .htaccess file

The default .htaccess file for WordPress is:

# BEGIN wordpress

RewriteEngine On
RewriteBase /
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule . /index.php [L]

# END wordpress

This means that if a file is requested from the server, if it is not a file that exists in the server’s folder directory (!-f) and if it is not a directory that exists in the server’s folder directories (!-d) then pass the request onto index.php. This way, WordPress will handle both customized URLs (such as those used for SEO) and also 404 File Not Found errors.

If you set up a password protected folder in a directory included in a WordPress install, all of a sudden WordPress takes over that folder and returns a 404 page, like the file doesn’t exist.

This happens because of a little ‘gotcha’ in the apache configuration. Luckily it is an easy fix.

In the password protected folder’s .htaccess file, you may already have the entries to ask for password access. Before all of that, place the following line:

ErrorDocument 401 /401.html

Then create a 401.html in the main folder, with any text, for example:

PASSWORD PROTECTED FOLDER - Please enter the correct username/password.

Voila, you can now enter your password protected folder again.

There is another workaround this little error, but since it involves editing the main wordpress .htaccess file, it can be overridden during an upgrade.

Solution found on: http://www.webmasterworld.com/apache/3688208.htm

Simple Redirect Subdomain to a Directory

If you are running Apache for your web server, and mod_rewrite is installed (this is a pretty typicaly module on all installations) this is actually pretty easy.

RewriteEngine on
RewriteCond %{HTTP_HOST} ^subdomain\.yourdomain\.com
RewriteRule ^(.*)$ http://www\.yourdomain\.com/subdomain/$1 [L]

Add this code to your apache configuration file – the easiest location is in the .htaccess file in your root web directory. This should redirect the browser with a 302 Found message.

You can do some pretty fancy things with mod_rewrite, but this is simple and gets the job done!

Microsoft Office 2007 New Formats and your Apache server

If you upload those new fancy-shmancy file formats to your web server – namely .docx, .pptx and .xlsx – and you are running Apache; chances are that your web server doesn’t know how to serve those files because they are unknown file formats. Your browser may try to download them as a .zip file (IE) or just display the binary format (Firefox) which ends up looking like jibberish with some XML data.

It’s relatively easy to fix this problem, you just need to tell apache how to handle those files.

Find the file mime.types, this may be in /etc/ or in /etc/httpd/conf/.

Add the following line to this file:

application/vnd.openxmlformats docx pptx xlsx

In one line:

echo "application/vnd.openxmlformats docx pptx xlsx" >> /etc/httpd/conf/mime.types

Restart both Apache and your web browser. Clearing the cache doesn’t work (I learned the hard way :))

Your file should now be downloaded properly to your PC.

Use PHP, GD and .htaccess to Watermark All Images in a Directory

The goal here is to watermark all images in a certain directory, except for thumbnails or other selection. You can either do this on each file prior to placing on your webserver – which is probably wise for CPU load issues – but let’s just say you want to do this for all files served in a single directory dynamically, a gallery for example.

The first step is to create a .png file with transparency which holds your watermark image. For this exercise, I’ve created this image:

tbwm.png

(I’ve added the border to stand the image out from the background of the page).

Here is the original image we are going to test with:

boratwow.jpg

After we have our watermark and sample image, we need to write a php file to use PHP’s GD function to apply this image to our original image. The particular function we use is imagecopy(). Here is the code I am using, I name it w.php:


This file is placed in the images directory.

Also in the images, create an .htaccess file with the following code:

RewriteEngine on
RewriteRule ^([^thumb].*\.[jJ].*)$ /com/gallery/w.php?i=$1

This tells the web server that instead of serving jpg files out of this directory, that we should instead process the filename with w.php and then serve to the browser. It also adds in a clause that if it starts with thumb_, that it will not run on this file. This is so it does not run on thumbnails.

Here is the resulting image, with watermark! This is served right out of an image directory with no watermark on the original picture:

Borat with watermark from php