Work the Shell - How Do People Find You on Google?

One of my favorite sequences in Linux is sort | uniq -c | sort -rn, and that's going to come into play again here. What does it do? It sorts the input alphabetically, then compresses duplicate lines with a preface count of how many matches are found. Then, it sorts that result from greatest matches to least. In other words, it takes raw input and converts it into a numerically sorted summary.

This sequence can be used for lots and lots of tasks, including figuring out the dozen most common words in a document, the least frequently used filename in a filesystem, the largest file in a directory and much more. For our task, however, we simply want to pore through the log files and figure out the most frequent searches that led people to our Web site:

#!/bin/sh

ACCESSLOG="/var/logs/httpd.logs/access_log"

grep 'google.com/search' $ACCESSLOG | \
  awk '{print $11}' | \
  cut -d\? -f2 | cut -d\& -f1 | \
  sed 's/+/ /g;s/%22/"/g;s/q=//' | \
  sort | \
  uniq -c | \
  sort -rn | \
  head -5

And the result:

$ sh google-searches.sh
 154 hl=en
  42 sourceid=navclient
  13 client=safari
   9 client=firefox-a
   3 sourceid=navclient-ff

Hmmm... looks like there's a problem in this script, doesn't there?

I'm going to wrap up here, keeping you in suspense until next month. Why don't you take a stab at trying to figure out what might be wrong and how it can be fixed, and next month we'll return to this script and figure out how to make it do what we want, not what we're saying it should do!