An Ajax-Enhanced Web-Based Ethernet Analyzer

Combine Ruby, Ajax and bash with CGI scripts to monitor server-bound processes.

I've spent the past six months or so playing with Ruby. I blame the July 2006 issue of Linux Journal for this hiatus from my programming language of choice, Perl, as that issue opened my eyes to the possibilities of using Ruby as a serious tool. I still love, use and teach Perl, but I'm spending more and more time programming in Ruby.

I follow the same process when learning any new programming technology: I identify a good book and work through it, and then start to use the language to build some of the things I love to build with Perl. Identifying the book was easy. The second edition of Programming Ruby by Dave Thomas (known as The PickAxe) is as good an introduction as you are likely to find for any programming language, not just Ruby. Once I'd worked my way through The PickAxe—creating a Ruby tutorial as I went along (see Resources)—I was itching to write some real code. I started with a type of tool that I enjoy building with Perl: a custom Ethernet analyzer.

Does the World Really Need Another Ethernet Analyzer?

At this point, probably more than a few readers are saying to themselves: why bother creating an Ethernet analyzer when tcpdump and Ethereal/Wireshark already exist? Those solutions are excellent tools—which I use a lot—but, I'm often looking to build something that involves additional processing plus the capturing and decoding of Ethernet packets, and this customization invariably involves resorting to custom code. Luckily, it turns out that the technology that underpins both tcpdump and Ethereal/Wireshark—as well as the hugely popular Snort IDS—is available as a library and that a number of language bindings exist for it. The packet capturing library, called libpcap, is available from the same project that brought the world tcpdump and can be downloaded with ease from the Web. In fact, it may well be included within your distribution's package management system; it is if you are running a recent release of Ubuntu (as I am). Obviously, the intrepid programmer can use C with libpcap, but—let's be honest here—life's far too short to work at the C level of abstraction when something more agile is needed. Thankfully, Perl provides an excellent set of modules that work with libpcap, and I devote one-sixth of my first book to discussing the Perl technology in detail. To my delight, and after a little digging around, I also found a set of Ruby classes that interface to libpcap (see Resources).

Creating a Custom Ethernet Analyzer with Ruby

In order to test the libpcap technology for real, I decided to use Ruby to redevelop a tool I created with Perl a number of years ago, which I wrote about within the pages of The Perl Review (see Resources). My Perl tool, called wdw (short for who's doing what?), analyzes requests made to a LAN's DNS service and reports on the site names for which the clients are requesting DNS resolutions. In less than 100 lines of Perl code, I'd written a functioning and useful DNS Ethernet analyzer. I wondered how using Ruby would compare.

Now, I present the 20 or so lines of Ruby I used to re-create wdw (for the entire program, see Listing 1). Do not interpret my numbers as any attempt to claim that Ruby can do what Perl does in one-fifth the number of lines of code. It cannot. It is important to note, however, that Ruby's interface to libpcap is significantly more abstract than the one offered by Perl, so Ruby does more in a single call than Perl does, but that has more to do with the choices made by the creators of each language's libpcap binding, as opposed to any fundamental language difference.

Before executing this code, download and install Ruby's libpcap library. Pop on over to the Ruby libpcap Web site (see Resources), and grab the tarball. Or, if you are using Ubuntu, use the Synaptic Package Manager to download and install the libpcap-ruby1.8 package. If a distribution package isn't available, install the tarball in the usual way.

You also need a Ruby library to decode DNS messages. Fortunately, Marco Ceresa has been working hard at porting Perl's excellent Net::DNS module to Ruby, and he recently released his alpha code to RubyForge, so you need that too (see Resources). Despite being alpha, Marco's code is very usable, and Marco is good at releasing a patched library quickly after any problems are brought to his attention. Once downloaded, install Marco's Net::DNS library into your Ruby environment with the following commands:

tar zxvf net-dns-0.3.tgz
cd net-dns-0.3
sudo ruby setup.rb

My Ruby DNS analyzer is called dns-watcher.rb, and it starts by pulling in the required Ruby libraries: one for working with libpcap and the other for decoding DNS messages:

#! /usr/bin/ruby -w

require 'pcap'
require 'net/dns/packet'

I can tell my program which network connection to use for capturing traffic, or I can let libpcap-ruby work out this for me. The following line of code lets Ruby do the work:

dev = Pcap.lookupdev

With the device identified (and stored in dev), we need to enable Ethernet's promiscuous mode, which is essential if we are to capture all the traffic traveling on our LAN. Here's the Ruby code to do this:

capture = Pcap::Capture.open_live( dev, 1500 )

The open_live call takes two parameters: the device to work with and a value that indicates how much of each captured packet to process. Setting the latter to 1500 ensures that the entire Ethernet packet is grabbed from the network every time capturing occurs. The call to open_live will succeed only if the program has the ability to turn on promiscuous mode—that is, it must be run as root or with sudo. With the network card identified and ready to capture traffic, the next line of code applies a packet capturing filter:

capture.setfilter( 'udp port 53' )

I'm asking the libpcap library to concern itself only with capturing packets that match the filter, which in this case is Ethernet packets that contain UDP datagrams with a source or destination protocol port value of 53. As all Net-heads know, 53 is the protocol port reserved for use with the DNS system. All other traffic is ignored. What's cool about the setfilter method is that it can take any filter specification as understood by the tcpdump technology. Motivated readers can learn more about writing filters from the tcpdump man page.

A constant is then defined to set how many captured packets I am interested in, and then a timestamped message is sent to STDOUT to indicate that the analyzer is up and running:

NUMPACKETS = 50
puts "#{Time.now} - BEGIN run."

The libpcap-ruby library contains the loop iterator, which provides a convenient API to the packet capturing technology, and it takes a single parameter, which is the number of packets to capture. Each captured packet is delivered into the iterator's body as a named parameter, which I refer to as packet in my code:

capture.loop( NUMPACKETS ) do |packet|

Within the iterator, the first order of business is to decode the captured packet as a DNS message. The Packet.parse method from Marco's Net::DNS library does exactly that:

dns_data = Net::DNS::Packet.parse( packet.udp_data )

With the DNS message decoded, we can pull out the DNS header information with a call to the header method:

dns_header = dns_data.header

For my purposes, I am interested only in queries going to the DNS server, so I can ignore everything else by checking to see whether the query? method returns true or false:

if dns_header.query? then

Within the body of this if statement, I print out the IP source and destination addresses, before extracting the IP name from the query, which is returned by calling the dns_data.question method. Note the use of a regular expression to extract the IP name from the query:

print "Device #{packet.ip_src}
 ↪(to #{packet.ip_dst}) looking for "
question = dns_data.question
question.inspect =~ /^\[(.+)\s+IN/
puts $1
STDOUT.flush

The program code concludes with the required end block terminators, and then the capture object is closed, and another timestamp is sent to STDOUT:

    end
end
capture.close
puts "#{Time.now} - END run."
______________________

Comments

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

Nice!

millerb's picture

Thanks for the excellent follow-up to your TPR article. There are plenty of use cases for a network analyser that do not require promiscuous mode - which would simplify the approach. In general, I prefer not to operate in promiscuous mode since I am only interested in monitoring point-to-point traffic.

As a consultant who frequently performs network analysis and tuning I am keenly interested in a solution like this. While I like the Ajax/Apache approach my customers aren't likely to be crazy about me installing the entire kit on their boxes. Wrapping this approach into one executable would be ideal. Or, simply installing an agent which would send the results to an Apache instance running on my laptop :-)

Cheers

Wrapping this approach into one executable would be ideal

barryp's picture

Thanks for the positive comment. If all you need is the results, all your customers need is Ruby (or Perl) installed on their boxes, with a little script that wakes up every now & then and sends stuff to you. My article simply used the analyzer as a way to generate a lot of server side data which allowed me to demo the Ajax solution.

--
Paul Barry
IT Carlow, Ireland
http://glasnost.itcarlow.ie/~barryp

Paul Barry

broken link

Anonymous's picture

Thanks for this nice article!

There is only one thing I want to add:
The above link to the sources appears to be dead...
Here is the fixed one:
ftp://ftp.linuxjournal.com/pub/lj/listings/issue157/9614.tgz

Webinar
One Click, Universal Protection: Implementing Centralized Security Policies on Linux Systems

As Linux continues to play an ever increasing role in corporate data centers and institutions, ensuring the integrity and protection of these systems must be a priority. With 60% of the world's websites and an increasing share of organization's mission-critical workloads running on Linux, failing to stop malware and other advanced threats on Linux can increasingly impact an organization's reputation and bottom line.

Learn More

Sponsored by Bit9

Webinar
Linux Backup and Recovery Webinar

Most companies incorporate backup procedures for critical data, which can be restored quickly if a loss occurs. However, fewer companies are prepared for catastrophic system failures, in which they lose all data, the entire operating system, applications, settings, patches and more, reducing their system(s) to “bare metal.” After all, before data can be restored to a system, there must be a system to restore it to.

In this one hour webinar, learn how to enhance your existing backup strategies for better disaster recovery preparedness using Storix System Backup Administrator (SBAdmin), a highly flexible bare-metal recovery solution for UNIX and Linux systems.

Learn More

Sponsored by Storix