At Your Service—Job Scheduling for Linux
Drawing from the communications world, services fall in one of two broad categories, immediate or deferred. Immediate communication services include teleconferencing, on-line chat, etc.; while deferred services include e-mail and fax. This column presents an overview of the Linux at utility that allows for the deferred (or scheduled) execution of a program.
Of course, most programs need to be launched in an immediate, do-it-now mode. Still, it's often desirable and perhaps required to schedule the execution of a program (or job, in ancient Geek) at a particular time for a variety of practical reasons. Consider any of the following scenarios:
A development group has developed a script that rebuilds their source tree and updates the application's staging area. The script is completely mechanized, but it is very time- and CPU-intensive. As a practical matter, the group decides this job should be scheduled to run in the late evening when it would be competing with fewer users and processes.
A high-quality color printer/plotter is in high demand during regular business hours. Multiple times a week, a project manager uses the printer to reprint a quickly evolving project plan of 200+ pages. To avoid monopolizing the printer during its peak usage period, the project manager decides the print job should be run in the early morning—before most users are active.
A project leader wants to distribute meeting reminder/agenda notes via e-mail to participating staff members four hours before a meeting scheduled for next Friday at 4:00 P.M.
A system administrator (SA) has been receiving reports of extremely poor system performance at the start of the business day. The cause of the degrading performance is not obvious. Rather than comb through numerous system logs, the SA decides to schedule the execution of a program that captures system load information. The job is to be executed the next day, every five minutes during the 9:00 A.M. hour.
An external data source used to update a local data warehouse becomes available at 6:00 P.M. every Friday. If the lengthy update procedure does not complete successfully, it's to be retried an hour later. Otherwise, it is rescheduled for the next Friday.
The scheduling requirements outlined in each of the above scenarios can be addressed by the at utility. Furthermore, these scenarios represent a broad range of the most common applications for at. The at utility is ideal for performing CPU-intensive tasks when little user-processing demand is present on the system, utilizing scarce resources at a time when they're generally more available, distributing reminders at a specific time, executing jobs that need to run at a time when the user has no intention of being actively connected to the system and executing jobs that are dependent on resources becoming available at a specific time.
at is actually a collection of related programs that allow Linux users to schedule and manage the execution of deferred jobs. If you're thinking at is only suitable for noninteractive tasks, you are correct. Programs scheduled through at are expected to be capable of running as background processes, since at does not associate user-display devices with executing jobs. Users generally find the at command-line interface to be quite intuitive. Yet, it is expressive enough to meet any conceivable scheduling requirement. Before presenting examples of the command-line interface, let's summarize the basic services provided by at. It allows for scheduling the execution of a job at a specific time/date, displaying information about jobs currently scheduled, canceling a job currently scheduled and administrating the list of users that have permission to use at programs.
The examples presented here assume you have a basic familiarity with shell command execution and Bourne shell syntax. Each of the examples were tested on a Red Hat Linux 7 (2.3 kernel) platform. However, I would expect the examples to work as presented on all common distributions.
Obviously, the most fundamental capability offered by at is that of scheduling the execution of a program at some later time. Generally installed at /usr/bin/at, the basic command-line syntax for the at program is as follows:
at [options] TIME [DATE] < bourne-shell-file
This syntax implies the following alternate ways of invoking at would work as well:
cat bourne-shell-file | at [options] TIME [DATE] at [options] TIME [DATE] ## Anything normally accepted by the bourne shell ## interpreter will be accepted. Terminate with ## Ctrl-D (^d) statement 1 statement 2 statement 3 statement 4 ^dThe first significant detail here is the proper notation for the time and date. The date is always optional. When it's omitted from the command line, the date is assumed to be the next day the specified time will occur, that is, either today or tomorrow. For example, if the time is specified as 1:30 P.M. and it's already 4:30 P.M., the job will be scheduled for the next day at 1:30 P.M. at accepts a time/date notation that extends the POSIX.2 standards. That notation may be documented on your installation under the /usr/doc directory in a file called timespec.
Let's learn through some examples. The time/date specification examples in Table 1 will give you a good idea of what the at program will accept.
As shown, the at program accepts a fairly rich and intuitive notation. It attempts to interpret the time/date specified by parsing its command-line arguments from left to right. If its time/date specification is violated, a “Garbled time” error diagnostic is displayed< and the program terminates. It usually provides a terse hint as to why it became troubled. For example, review the following invocation attempt below:
$ at 6am Mar 32 Error in day of month. Last token seen: 32 Garbled time $
The following invocation succeeds at scheduling a job. Consider the third usage scenario outlined above for this submission:
$ at 1pm friday warning: commands will be executed using /bin/sh STAFF="moe larry curly" cd mail mail -s"Meeting Reminder" $STAFF < friday_agenda.txt ^d job 6 at Fri Apr 13 13:00:00 2001 $If you're following closely, some questions should begin to formulate. Do the standard output and standard error streams of a scheduled job get captured somewhere? All standard output and standard error diagnostics are captured by the at service (/usr/sbin/atd program) and e-mailed to the submitting user when the scheduled job finishes. The e-mail will appear with a subject heading such as the following:
Subject: Output from your job 17
Are we limited to scheduling Bourne shell scripts? As the feedback from the example above indicates, at uses the system Bourne shell program (/bin/sh) to interpret the user-provided program statements. Therefore, anything you would type at a Bourne shell prompt would be valid, including a Bourne shell script or the launch of any executable found in your environment, even a different language interpreter. Review the following example for scheduling the execution of a Perl script:
$ at 6pm tomorrow warning: commands will be executed using /bin/sh perl /home/moe/perls/script1.pl ^d job 28 at Wed Apr 18 18:00:00 2001 $
What system users/groups get assigned to the scheduled process? What attributes of the job submission environment are preserved and carried over into the job execution environment? Discovering how a utility provides a service behind the curtains is usually not of interest to most users. In the case of at, however, an actual example of what the utility does is fairly simple to follow and shows what's possible, what isn't and why. Every scheduled job results in a generated Bourne shell script placed under the /var/spool/at/ directory. These generated scripts are comprised of the following sections:
Program comments that provide some clues as to which user is notified when the job completes. Also, the effective user and group ID assigned to the job is specified.
umask setting dictates how new files/directories should be created.
A full listing of environment variable assignments at the time the job was submitted (except those related to display devices, such as TERM and DISPLAY).
The current directory changes to where the user was when the task was scheduled. If that directory does not exist when the job executes, the job aborts and an e-mail notification is sent.
An appended copy of the program text that was submitted to at.
With this in mind, Listing 1 is an abbreviated example of a script generated on behalf of a job scheduled by our user Moe. To complete our command-line interface description of the at program, see Table 2 for some of its more useful command-line options.
The following is another example of scheduling a job with at:
$ at -mf /home/curly/shells/program1 6pm tomorrow warning: commands will be executed using /bin/sh job 29 at Thu Apr 12 18:00:00 2001 $
Two other programs are tied to the basic at program: atq and atrm. As you might guess, atq lists jobs currently scheduled while atrm cancels one or more specified jobs. While the root user is capable of viewing and canceling any and all jobs, nonroot users can only view and cancel what they schedule.
Listing 2 shows an example of the output displayed by atq, as viewed by the root user (I inserted the field labels for better readability—unfortunately the Linux implementation of atq does not provide them).
Rank is a unique sequence value used to identify scheduled jobs to other at programs. In addition to the scheduled time and the submitting user, a queue value is listed. Unless the user specified otherwise, jobs are placed on the “a” queue. (Reference the on-line manual page for the implications of specifying an alternate queue value to the at program—it essentially controls the runtime priority of the job.) atq qualifies currently running jobs with a “=” value in the queue field. So job 17, seen in Table 3, is currently executing.
What if a user submits a job and then realizes either the time and/or program is incorrect? The atrm utility can be used to remove one or more scheduled jobs. For example, the root user could cancel the first and second jobs listed in Table 3 with the following command:
atrm 18 19
atrm provides no feedback. A subsequent atq listing would then display a queue like the one in Listing 3.
If your attempt to use one of the at programs produces the following diagnostic: “You do not have permission to use at”, you need to contact your local SA. Linux SAs can manage at with a fair amount of flexibility.
As you might expect, the root user has absolute permission to use the at utility and can grant the same permission to nonroot users. Two system files, /etc/at.allow and /etc/at.deny, control access to the at utility. Table 3 shows how their presence and content determines users' permission on a given system.
Note that if the /etc/at.allow files exist, /etc/at.deny is completely ignored. Users are identified in both files by their Linux login, each appearing on a separate line. at does not provide a command-line utility to control the content of these files. SAs generally select their favorite text editor and manually edit the files as needed. This hardly can be considered a shortcoming, though, given the likely infrequency of change.
To summarize, nonroot users can be explicitly or implicitly assigned or denied permission to use at. SAs either can choose to manage access to at by exclusion or inclusion. Select the approach that makes the most sense for your particular installation. For example, a highly sensitive production site probably should be managed based on inclusion (i.e., nonroot users do not have permission unless it's explicitly granted—the /etc/at.allow file exists). Conversely, the Linux default configuration might be fine for most development/test environments (i.e., nonroot users have permission unless it's explicitly denied—/etc/at.allow does not exist and /etc/at.deny has zero or more entries).
Collectively, the at programs offer an intuitive way to manage the deferred execution of applications. Despite its simplicity and usefulness, the at utility is often ignored by Linux administrators and developers. Other less frequently used at command-line options exist that I chose not to cover here. I encourage you to review the at manual page by typing man at at your favorite shell's prompt to review all details. Also, most Linux overview books provide some coverage of at and similar programs, such as O'Reilly's Linux in a Nutshell by Ellen Siever, et. al.