As the Log Scrolls By...

Gasson shows how a few tweaks to Apache's httpd.conf file can provide a colorful web log file.

As a web hosting company, there are times when it's vitally important to see what our Apache web server is dishing up to the outside world at any one time—and to see this as quickly as possible.

Just as a system administrator needs to be able to monitor system log files, a web administrator should be able to do the same with web logs. Noting the number of utilities that display system log information in real time, I was sure there'd be similar programs for monitoring web logs. After a search through Freshmeat.net and other on-line resources, I didn't find anything that met all of my needs. Several came close, but most would only monitor one file, and the few that monitored several files would leave me lost trying to wade through tabs for the 30 or so logs that I watch.

The solution I found isn't in one program, but in making a few changes in Apache's httpd.conf file I found I could have a specialized disposable log file containing only the information I require, for all of our web hosts. The “disposable” monitoring log is then displayed using colortail (with additional configuration settings) on an external monitor in our workshop. I can see at a glance which of our hosts have current web activity, where the traffic is coming from and the pages that are being accessed. This has also allowed us to deal quickly with problems such as script kiddies and rogue search engine robots. The system has worked so well for us that we added system logging to it as well.

httpd.conf Changes

In addition to the LogFormat for general logging, I added a new format labeled “webmonitor”:

LogFormat "[%v] %h %u \"%r\"%>s%b\n\"%{Referrer}i\" \"%{User-Agent}i\"%t" webmonitor

This displays log information with the Referrer and User-Agent on a second line, making it clearer to read. The log file could be in any format—even the “common” one we use for standard logging. I decided to change it for purposes of clarity and because Apache is flexible enough to allow this.

Since graphic files such as GIFs, JPEGs or PNGs files can clutter up the display, I exclude them by adding the following three lines to the general log section in httpd.conf:

SetEnvIf Request_URI \.gif$ unwanted
SetEnvIf Request_URI \.jpg$ unwanted
SetEnvIf Request_URI \.png$ unwanted

We use name-based virtual hosts, and each host has their own <VirtualHost></VirtualHost> container. In addition to their permanent log file, we add an additional CustomLog command for our webmonitor file for each of our hosts. For example:

<VirtualHost someisp.com>
...
CustomLog /var/log/httpd/someisp.com-access_log combined
CustomLog /var/log/httpd/webmonitor_log webmonitor env=!unwanted
...
</VirtualHost>
Our addition is:
CustomLog /var/log/httpd/webmonitor_log webmonitor env=!unwanted
/var/log/httpd/webmonitor_log is the path and filename for our monitoring log file, and Apache will create it for us at startup if it doesn't already exist. webmonitor is the name of our custom format log defined in the LogFormat section above. env=!unwanted sets Apache so it doesn't log any items we've listed in the SetEnvIf lines (the .gif, .jpg and .png file extensions). This way we don't see graphic file requests but we do see all others.

Adding System Log Information

The ability to see what's happening on the server with an external monitor proved so useful that we also included system logging information in the same file. To do this, we edited /etc/tem syslog.conf to include the following command:

kern.*;authpriv.*;*.crit;*.error;*.warning;*.emerg /var/log/httpd/webmonitor_log
Colortail

Colortail was written by Joakim Andersson (pt98jan@student.hk-r.se) and is available from www.student.hk-r.se/~pt98jan/colortail.html under the GNU Public License.

While we could simply tail the webmonitor log file, adding color to the display is a nice touch and gives us an indication of which web host is seeing activity even if we happen to be on the other side of the workshop.

Colortail comes with several sample configuration files; none really suited web logs, although conf.xferlog comes close. After a bit of tweaking, this is the format we've been using. It's a hybrid as it includes both web and system-log-related items.

Listing 1. colortail.conf

______________________

Webinar
One Click, Universal Protection: Implementing Centralized Security Policies on Linux Systems

As Linux continues to play an ever increasing role in corporate data centers and institutions, ensuring the integrity and protection of these systems must be a priority. With 60% of the world's websites and an increasing share of organization's mission-critical workloads running on Linux, failing to stop malware and other advanced threats on Linux can increasingly impact an organization's reputation and bottom line.

Learn More

Sponsored by Bit9

Webinar
Linux Backup and Recovery Webinar

Most companies incorporate backup procedures for critical data, which can be restored quickly if a loss occurs. However, fewer companies are prepared for catastrophic system failures, in which they lose all data, the entire operating system, applications, settings, patches and more, reducing their system(s) to “bare metal.” After all, before data can be restored to a system, there must be a system to restore it to.

In this one hour webinar, learn how to enhance your existing backup strategies for better disaster recovery preparedness using Storix System Backup Administrator (SBAdmin), a highly flexible bare-metal recovery solution for UNIX and Linux systems.

Learn More

Sponsored by Storix