Secure Logging Over a Network
Undoubtedly, one of the most important tasks for keeping a networked system safe is monitoring its activity. Most system programs today communicate with syslogd noting important events (such as an su request). Also, the kernel can note hardware failures and other things on that level. Of course, when monitoring for possible break-ins or trying to track down the path an intruder used to violate a system, the logs can be a precious resource. However, it is evident that an intruder could manipulate system logs if he were able to get the necessary permissions, thus completely fooling any attempt to track down the intrusion, even to the extreme point of making it hard to spot.
A first step toward solving the problem of keeping the system logs on the system itself was made many years ago when the ability to send logging messages over the network to another host was added to syslogd. Sending the needed log parts to another host, which is quite secure (a dedicated machine, for example), was actually a half-solution to the problem. Using a trivial protocol like UDP to transmit data over networks made it very easy to spoof, making it possible for the intruder to add malicious messages or even try to create a denial-of-service attack. Again, all data is sent through the network in clear form, making it simple to sniff, so an intruder could get more details about the system before attempting a break-in.
A solution we developed to solve this problem is having the standard syslogd facility use the secure shell (ssh) package to forward the logs, using authentication and encryption, to another computer. This way, the system we want to keep logged won't require special changes, and the system that will maintain the logs will not require another running syslogd. It will require sshd (secure shell dæmon) running with some normal system tools on a normal user account.
We will present two technically similar but conceptually different methods, with some pros and cons, of performing this important task. The first could be compared to a “push” technology; that is, the machine that wants to log its data on an outside machine will actually force data to go on the other side. The second method will use a “pull” technology, since the remote client will try to retrieve the logs from the desired machine.
Since our solutions use the standard syslog facilities to grab messages, you need a syslog dæmon that supports logging output to named pipes. Most recent syslog dæmons have this (check the man page for support, or try it directly). It is very important that you have a syslogd that opens pipes in non-blocking mode.
Older syslogs used to open the pipes in blocking mode, so if the named pipe was full (in our context, for example, if the connection had been broken for some time), then the whole syslog process stopped while waiting for the pipe to have some space available. This, of course, could have dramatic consequences, since all the logging, not just the remote one, would have been compromised. As a practical test to see whether your syslogd is handling pipes correctly, configure the whole system, then try writing in the pipe until it reaches the maximum dimension, usually 4KB. When the connection is down, check whether syslog is still logging something or if it seems to be frozen. Anyway, we suggest getting the latest version you can find (which should correct other bugs as well) and installing it on your system (at the moment, the latest version is 1.3-31). The more advanced syslog-ng package should work as well with our solution. Apart from this, you will need a Perl interpreter, the tail command from the textutils package, and of course, the secure shell package.
The theory behind this “push” solution is quite straightforward. A Perl script will run in the background, and will connect (using ssh) to the host where we wish to keep the copy of the logs. The Perl script will then monitor a named pipe we will create, and send anything it will get from the pipe over the secure connection to the other side, where we will put everything in a file. Instructing syslog to send the desired logging events to our named pipe will make the work complete.
Let's give a brief technical description of the source code (see Listing1). Once started, the script will open a local log (defined in $local_log) to report failures (this can be disabled by undefining the variable), then it will try to connect to the remote host (defined in $host). The connection subroutine opens the ssh command to the remote side, specifying the user name to be used, whether to use compression and to establish a quiet connection. As you can see, ssh will ask the remote end to execute a cat of a given file, so everything will be stored there for further use. Once connected, the script will open the local named pipe we created, then forward everything to the other side, where the cat command will store it. Of course, if the secure connection should die for some reason (technically, if the print on the file handle should fail), the script will try to establish the connection again.
As a last feature, the script will send a timestamp each defined time interval to the other end, which could be useful for tracking down when the last message actually came from the host if something serious has happened. In fact, the script can detect that the connection went down each time it sends something over the network. By sending a timestamp to the FIFO (i.e., first-in/first-out pipe), it will automatically trigger the check that the connection is alive at that moment.
Practical Task Scheduling Deployment
One of the best things about the UNIX environment (aside from being stable and efficient) is the vast array of software tools available to help you do your job. Traditionally, a UNIX tool does only one thing, but does that one thing very well. For example, grep is very easy to use and can search vast amounts of data quickly. The find tool can find a particular file or files based on all kinds of criteria. It's pretty easy to string these tools together to build even more powerful tools, such as a tool that finds all of the .log files in the /home directory and searches each one for a particular entry. This erector-set mentality allows UNIX system administrators to seem to always have the right tool for the job.
Cron traditionally has been considered another such a tool for job scheduling, but is it enough? This webinar considers that very question. The first part builds on a previous Geek Guide, Beyond Cron, and briefly describes how to know when it might be time to consider upgrading your job scheduling infrastructure. The second part presents an actual planning and implementation framework.
Join Linux Journal's Mike Diehl and Pat Cameron of Help Systems.
Free to Linux Journal readers.View Now!
|The Firebird Project's Firebird Relational Database||Jul 29, 2016|
|Stunnel Security for Oracle||Jul 28, 2016|
|SUSE LLC's SUSE Manager||Jul 21, 2016|
|My +1 Sword of Productivity||Jul 20, 2016|
|Non-Linux FOSS: Caffeine!||Jul 19, 2016|
|Murat Yener and Onur Dundar's Expert Android Studio (Wrox)||Jul 18, 2016|
- Stunnel Security for Oracle
- The Firebird Project's Firebird Relational Database
- Murat Yener and Onur Dundar's Expert Android Studio (Wrox)
- SUSE LLC's SUSE Manager
- Managing Linux Using Puppet
- My +1 Sword of Productivity
- Non-Linux FOSS: Caffeine!
- Doing for User Space What We Did for Kernel Space
- SuperTuxKart 0.9.2 Released
- Google's SwiftShader Released