Home ›

Work the Shell - Analyzing Log Files

Issue 150

From Issue #150
October 2006

Oct 01, 2006  By Dave Taylor
 in
Ever wondered what your Web server is doing, but find that you don't have a stats or analytics package installed? In fact, analyzing log files is a perfect task for the Linux command line and, by extension, shell scripts too.

(Yes, yes, I know that the URLs on this site are ridiculously long!)

Now, finally, I can see that the articles about the Motorola RAR phone, AOL screen names and Sony PSP are the most popular articles on the site. Remember, this is a slice for only about six hours too, so the RAZR article is actually being requested an impressive once a minute or so. Popular indeed!

I'm going to stop here now that you've had a taste of how basic Linux commands can be combined to extract useful and interesting data from an Apache log file. Next month, we'll look at one more statistic: how much aggregate data we've transferred. Then, we'll start looking at how to build a shell script that does these sorts of calculations with ease.

Dave Taylor is a 26-year veteran of UNIX, creator of The Elm Mail System, and most recently author of both the best-selling Wicked Cool Shell Scripts and Teach Yourself Unix in 24 Hours, among his 16 technical books. His main Web site is at www.intuitive.com.

______________________

Dave Taylor has been hacking shell scripts for over thirty years. Really. He's the author of the popular "Wicked Cool Shell Scripts" and can be found on Twitter as @DaveTaylor and more generally at www.DaveTaylorOnline.com.

White Paper
More Resources
Fabric-Based Computing Enables Optimized Hyperscale Data Centers

Today’s modular x86 servers are compute-centric, designed as a least common denominator to support a wide range of IT workloads. Those generic, virtualized IT workloads have much different resource optimization requirements than hyperscale and cloud applications. They have resulted in a “one size fits all” enterprise IT architecture that is not optimized for a specific set of IT workloads, and especially not emerging hyperscale workloads, such as web applications, big data, and object storage. In this report, you will learn how shifting the focus from traditional compute-centric IT architectures to an innovative disaggregated fabric-based architecture can optimize and scale your data center.

Learn More

Sponsored by AMD

White Paper
More Resources
Red Hat White Paper: Using an Open Source Framework to Catch the Bad Guy

Built-in forensics, incident response, and security with Red Hat Enterprise Linux 6

Every security policy provides guidance and requirements for ensuring adequate protection of information and data, as well as high-level technical and administrative security requirements for a system in a given environment. Traditionally, providing security for a system focuses on the confidentiality of the information on it. However, protecting the data integrity and system and data availability is just as important. For example, when processing United States intelligence information, there are three attributes that require protection: confidentiality, integrity, and availability.

Learn more about catching the bad guy in this free white paper.

Learn More

Sponsored by DLT Solutions