Using an SMS Server to Provide a Robust Alerting Service for Nagios

How to implement a Nagios-to-SMS service.
Apache

I initially had multiple overlapping “Directory” statements in the Nagios section of the Apache configuration file. The net effect was a “Permission denied” when the CGI was being run. I figured this out by using the method described below and by looking at the Apache access_log and error_log files.

Network Capture

If you think there is some communication problem with the script, you can monitor the traffic between Nagios, Apache and the iSMS by listening on the network. I used tcpdump to capture the HTTP traffic and see error messages:

% tcpdump -v -s 0 -w /tmp/cap host 192.168.1.50

In this example, I used the -v option for verbose output, the -s 0 option to capture as much of the packet as possible and the -w option to write the captured traffic to the /tmp/cap file. The “host” keyword indicates that I want all traffic to and from the IP address of the iSMS (192.168.1.50). I ran this command on the machine hosting both Nagios and Apache, so it should see all communication between these services and the iSMS. I then generated some SMS messages traffic by causing Nagios to send out a “PROBLEM” message, which I then acknowledged via my mobile phone. You should see the number following “Got” incrementing as packets are being captured:

tcpdump: listening on eth0, link-type EN10MB (Ethernet),
          capture size 65535 bytes
Got 22

I then interrupted the capture and converted the captured data to plain text:

% tcpdump -A -r /tmp/cap > /tmp/txt

The -A option writes out ASCII text, and the -r option reads capture data from a file. Examining the /tmp/txt file allows you to see the entire HTTP transaction between Nagios, the iSMS and the CGI script:


12:50:09.434851 IP nagios.46058 > smsgw.http:
    P 1:266(265) ack 1 win 46
    <nop,nop,timestamp 2801435752 1987587011>

GET /sendmsg?user=nagios&passwd=secret...text=ACKNOWLEDGEMENT...

12:50:09.501524 IP smsgw.http > nagios.46058:
    P 1:29(28) ack 266 win 6432
    <nop,nop,timestamp 1987587017 2801435752>

HTTP/1.0 200 OK
ID: 2078

In this capture, you can see that the sendsms.pl script invoked by Nagios (hostname nagios) has sent an HTTP GET to the iSMS (hostname smsgw) containing the Nagios “ACKNOWLEDGEMENT” message. The “ID: 2078” response from the iSMS back to Nagios indicates that the message has been queued for sending and that the ID for this SMS message is 2078. You also might note that the user name and password for the iSMS “nagios” account is being sent in the clear—not perfect, but I think this is a pretty low security risk, as this transaction is internal to the company network.

Firmware Version of iSMS

My original iSMS came with version 1.20 firmware. This worked fine with the original Perl script, but it had a problem in that it was somewhat “single user”. For example, if you happened to be logged in to the iSMS Web user interface while the check_smsfinder.pl script ran, it would return a bad status, and Nagios would create an alert for the device. Upgrading to the newer firmware fixed this problem, but broke the check_smsfinder.pl script. The Perl script has been updated, but the version of the script now is tied to the firmware version running on the iSMS.

Because this is a Perl script, it can be modified easily. If you do not like the format of the message being sent out by Nagios, you can change this in the Nagios “command” definition—for example, “notify-host-by-sms” and also change the Perl script to parse whatever format of a message you want to send back from your phone. The script authors have changed their message format over time to make it easier to parse, as problems were discovered with whitespace in service names and host alerts that would change format depending on whether the host definition contained an IP address (such as in the case of DHCP clients).

Conclusion

I am very pleased with how this alerting service has worked out. The iSMS has been solid since the moment I installed it, and the associated script has worked perfectly once I tweaked my Nagios setup to match it. I have high confidence that I will get alerts regardless of the nature of the problem.

Acknowledgements

Thanks to Birger Schmidt and his colleagues from NETWAYS GmbH (www.netways.de) for writing the original script, updating it and taking the time to review this article, and to Chris Reilley (www.reilleydesign.com) for Figure 2.

Eric Pearce is the IT Lead for AmberPoint, Inc., an Application Management and Governance software company based in Oakland, California. He has authored several books on UNIX and Windows system administration for O'Reilly & Associates.

______________________

White Paper
Linux Management with Red Hat Satellite: Measuring Business Impact and ROI

Linux has become a key foundation for supporting today's rapidly growing IT environments. Linux is being used to deploy business applications and databases, trading on its reputation as a low-cost operating environment. For many IT organizations, Linux is a mainstay for deploying Web servers and has evolved from handling basic file, print, and utility workloads to running mission-critical applications and databases, physically, virtually, and in the cloud. As Linux grows in importance in terms of value to the business, managing Linux environments to high standards of service quality — availability, security, and performance — becomes an essential requirement for business success.

Learn More

Sponsored by Red Hat

White Paper
Private PaaS for the Agile Enterprise

If you already use virtualized infrastructure, you are well on your way to leveraging the power of the cloud. Virtualization offers the promise of limitless resources, but how do you manage that scalability when your DevOps team doesn’t scale? In today’s hypercompetitive markets, fast results can make a difference between leading the pack vs. obsolescence. Organizations need more benefits from cloud computing than just raw resources. They need agility, flexibility, convenience, ROI, and control.

Stackato private Platform-as-a-Service technology from ActiveState extends your private cloud infrastructure by creating a private PaaS to provide on-demand availability, flexibility, control, and ultimately, faster time-to-market for your enterprise.

Learn More

Sponsored by ActiveState