swatch: Automated Log Monitoring for the Vigilant but Lazy
Once swatch is configured and running, we must turn our attention to the Goldilocks Goal: we want swatch to be running neither too hot (alerting us about routine or trivial events) nor too cold (never alerting us about anything). But what constitutes just right? There are as many different answers to this question as there are uses for UNIX.
Anyhow, you don't need me to tell you what constitutes nuisance-level reporting: if it happens you'll know it. You may even experience a scare or two in responding to events that set off alarms initially but turn out to be harmless nonetheless. Read the manual, tweak .swatch.rc and stay the course.
The other scenario, in which too little is watched for, is much harder to address, especially for the beginning system administrator. By definition, anomalous events don't happen too frequently, so how do you anticipate how they'll manifest themselves in the logs? My first bit of advice is to get in the habit of browsing your system logs often enough to get a feel for what the routine operation of your systems looks like.
Better still, tail the logs in real time. If you enter the command
tail -f /var/log/messages
the last 50 lines of the system log will be printed, plus all subsequent lines, as they're generated, until you kill tail with a Ctrl-C. This works for any file, even a log file that changes rapidly.
Another good thing you can do is to “beat up on” your system in one virtual console or xterm while tailing various log files in another. The tools we explored last month and the month before, Nessus and nmap, respectively, are perfect for this.
By now you may be thinking, “Hey, I thought the whole reason I installed swatch was so I wouldn't have to watch log files manually!” Nope. swatch minimizes, but does not eliminate, the need for us to parse log files.
Were you able to quit using your arithmetic skills after you got your first pocket calculator? No. For that matter, can you use a calculator in the first place unless you already know how to add, multiply, etc.? Definitely not. Same goes for log file parsing: you can't tell swatch to look for things you can't identify yourself, no more than you can ask for directions to a town whose name you've forgotten.
In the same vein, I urge you to not be complacent about swatch silence. If swatch's actions don't fire very often, it could be that your system isn't getting probed or misused often, but it's at least as likely that swatch isn't casting its net widely enough. Continue to scan through your logs manually from time to time to see if you're missing anything, and continue to tweak .swatchrc.
And don't forget to reconsider periodically the auditing/logging configurations of the dæmons that generate log messages in the first place. swatch won't catch events that aren't logged at all. Refer to the syslogd(8) man page for general instructions on managing your syslog dæmon and the man pages of the various things that log to syslog for specific instructions on changing the way they log events.
Practical Task Scheduling Deployment
July 20, 2016 12:00 pm CDT
One of the best things about the UNIX environment (aside from being stable and efficient) is the vast array of software tools available to help you do your job. Traditionally, a UNIX tool does only one thing, but does that one thing very well. For example, grep is very easy to use and can search vast amounts of data quickly. The find tool can find a particular file or files based on all kinds of criteria. It's pretty easy to string these tools together to build even more powerful tools, such as a tool that finds all of the .log files in the /home directory and searches each one for a particular entry. This erector-set mentality allows UNIX system administrators to seem to always have the right tool for the job.
Cron traditionally has been considered another such a tool for job scheduling, but is it enough? This webinar considers that very question. The first part builds on a previous Geek Guide, Beyond Cron, and briefly describes how to know when it might be time to consider upgrading your job scheduling infrastructure. The second part presents an actual planning and implementation framework.
Join Linux Journal's Mike Diehl and Pat Cameron of Help Systems.
Free to Linux Journal readers.Register Now!
- Stunnel Security for Oracle
- SourceClear Open
- Murat Yener and Onur Dundar's Expert Android Studio (Wrox)
- SUSE LLC's SUSE Manager
- My +1 Sword of Productivity
- Managing Linux Using Puppet
- Non-Linux FOSS: Caffeine!
- Tech Tip: Really Simple HTTP Server with Python
- Google's SwiftShader Released
- Parsing an RSS News Feed with a Bash Script