Managing your Logs with Chklogs

An introduction to a program written by Mr. Grimaldo to manage system logs.
Chklogs Configuration

The Chklogs package consists of an administrative program (chklogsadm), the main program (chklogs) and some Perl modules (Smtp.pm, Chklogs.pm, Interp.pm). For the second part of the configuration, we need the Chklogs configuration file (chklogs.conf) and, optionally, the resource file.

The Personal Resource File

The resource file (.chklogsrc) resides in your home directory. You need it if you wish to first test on a copy of your logs or if you don't have root permissions (therefore, not used for the system logs but for others). When Chklogs runs, it first gets configuration information that is coded in the scripts configuration variables (the configuration section is clearly marked in the source), then it looks for a Personal Resource File (PRF) and then the command-line options, in that order.

In the PRF you can override some variables from the script configuration, for example, the admin user and others. Here is an example provided in the distribution:

# Personal resource file for Chklogs 2.x
set ChklogsConf ~/devel/test/chklogs.conf
set ChklogsDb ~grimaldo/devel/test/.chklogsdb
set VarRun  ~/devel/test
set RelativePath MyOldLogs
set Admin  grimaldo
mode ignore on
set SyslogConf /etc/syslog.conf

You will notice it has a syntax similar to Tcl. That is, with the exception of the mode lines, everything between ignore on and ignore off is not interpreted. The Tcl/Tk GUI does have a dummy mode routine, but all variable settings are performed as this ignore mode is only meaningful in the Perl mode. Personally, I find the dollar sign in front of normal variables annoying. This same resource file (with some extensions) is used by the GUI. The PRF example from above does not show the variables that are used exclusively by the GUI. However, for the examples in this article, I will assume you are working on the actual logs and not a copy.

In the above script, all variables are overrides for the internal configuration variables. ChklogsConf indicates the location of the configuration file (introduced in the next section), ChklogsDb is the internal database (you must never edit or move this file around), and VarRun is the directory where the PID lock files are kept (including where to find Syslog's and Chklogs' PID file). RelativePath is used when you choose the local repository, and Admin is the e-mail address where the reports are sent when the mail command-line option is specified. Normally, I don't need this file when running Chklogs, but I do use it during testing as a non-privileged user.

The Configuration File

Listing 1 is a sample configuration file demonstrating the various features. This file contains three types of lines: comments/spacers, directives and specifications. It is divided into two major sections, the header and the body:

  • A directive line begins with the two characters #: and because this pattern is checked first, it is not confused for a comment.

  • A comment line is introduced by the # character. Note that you cannot have comments at the end of a specification line.

  • The specification line is any line that is not empty or a directive or comment. It defines the attributes for each of the logs you wish ChkLogs to manage.

The header is comprised of the comment lines that identify this configuration for a particular host, and the two directive lines which must appear in the order shown. The header can be created with either of the following two administrative commands:

chklogsadm newconf -global /var/log/Chklogs

or

chklogsadm newconf -local
The first directive (options) is used to set global options for this configuration. At present only two option names are recognized, and both deal with the Alternate Repository Feature explained earlier. The two acceptable options are global and local. The second directive (global) is also related to the Alternate Repository. It must be set to the root directory where the logs are going to be archived (i.e., those with “archive” attribute) by group hierarchy.

In the instruction section or body, we find mostly specification lines. These lines follow a particular syntax:

LogName Threshold Action [Parameter(s)]

The first field is a fully-qualified log name, the second is a threshold that can be a size or age. The size must be in bytes or, optionally, append the K qualifier for kilobytes. If you opt for an age, it must be in days by appending a D (e.g., 7d) or in months by appending an M (e.g., 2m). The qualifiers D, M and K are case insensitive. The log is always compared to the threshold, if it is reached then the specified “Action” is taken.

The Action field can have a value of archive, execute or truncate, which has no arguments, Chklogs uses the truncate system call to chop the file to a length of zero. Use this option if you don't care about this log or after you have used the log with either the archive or execute options as shown in Listing 1.

The archive action takes one parameter that indicates how many logs to keep archived before rotating/shuffling. If no parameter is given, an internal one is used (see documentation). Use this option, if you plan to look at this log at a later time.

The execute action is used to spawn a plug-out, that is, an external program that will operate on the log. This is useful if you have a particular log scanner that will filter the log and give you a human-readable report. You can specify as many parameters as you wish, the special parameter %L is replaced by Chklogs with the full name of the log and is used to inform the plug-out which file to process.

Listing 1 also shows a Logical Group definition. There, a group named NetNews is created so that, if a global alternate repository is used, there will be a NetNews directory under it in addition to the /common directory. For each group, you can associate an external program to execute in the plug-out interface. Parameters are allowed, but %L is meaningless in that context. Pre and Post specify which program to execute before and after the group has been processed respectively. A group definition always starts with the three directive lines in that order, then one or several logs and, finally, an empty line or a comment line. Do not include any of these in between the lines of the group definition.

In the case of the sample NetNews group, we want the daemon to stop using the logs before we touch them and restart (or continue) after we are finished. This is particularly useful for INN (which has a program to control the daemon) and HTTP. Chklogs handles syslog automatically; it knows what to do and how to do it.

Finally, notice that the last specification line refers to a directory as explained earlier in the Directory Lumping section.

______________________

White Paper
Linux Management with Red Hat Satellite: Measuring Business Impact and ROI

Linux has become a key foundation for supporting today's rapidly growing IT environments. Linux is being used to deploy business applications and databases, trading on its reputation as a low-cost operating environment. For many IT organizations, Linux is a mainstay for deploying Web servers and has evolved from handling basic file, print, and utility workloads to running mission-critical applications and databases, physically, virtually, and in the cloud. As Linux grows in importance in terms of value to the business, managing Linux environments to high standards of service quality — availability, security, and performance — becomes an essential requirement for business success.

Learn More

Sponsored by Red Hat

White Paper
Private PaaS for the Agile Enterprise

If you already use virtualized infrastructure, you are well on your way to leveraging the power of the cloud. Virtualization offers the promise of limitless resources, but how do you manage that scalability when your DevOps team doesn’t scale? In today’s hypercompetitive markets, fast results can make a difference between leading the pack vs. obsolescence. Organizations need more benefits from cloud computing than just raw resources. They need agility, flexibility, convenience, ROI, and control.

Stackato private Platform-as-a-Service technology from ActiveState extends your private cloud infrastructure by creating a private PaaS to provide on-demand availability, flexibility, control, and ultimately, faster time-to-market for your enterprise.

Learn More

Sponsored by ActiveState