Automating Tasks with EXPECT

As the System Administrator for 18 VAXs and 6 HP/UNIX machines, I am always looking for an easier way to do things. One thing I always do is write shell scripts for UNIX and command procedures for VMS to automate some of my tasks. If I could run them from one computer, I could administer the world from one place without logging on to each computer to run each task I have to perform.

I first tried using VAX/VMS command files in order to TELNET to the other machines. That method was quickly dropped, as the remote computer's I/O response arrived before my command file could do an input statement. Logons hung up quickly. I then tried to TELNET from a UNIX script—the same thing occurred. This type of frustration is why an administrator's job is tough.

Hanging out in a Barnes & Noble bookstore and leaning on the UNIX shelves is always a good pastime—you look good, and you help dust off the books. Leaning is how I discovered Exploring EXPECT, an interesting title for a book with a back cover stating “automate TELNET, FTP, passwd...”. I bought the book—yes, I judge a book by its cover (sometimes).

The excellent book, Exploring EXPECT by Don Libes (published by O'Reilly & Associates) contains everything you need to know about controlling any spawned process. You can spawn a game and interact with it as easily as a remote TELNET session. The book is fairly long and very detailed, but don't be intimidated. There is an easy way to learn to write EXPECT scripts; I'll get to it shortly.

To make a long story short, EXPECT is a toolkit for automating interactive programs, such as TELNET and FTP, written by Don Libes. EXPECT has to be the greatest asset to system administrators in all of history. Finally, I can easily write and execute TELNET logon scripts and do whatever I wish using a script file from one machine.

EXPECT reads commands from a script file, spawns a process like TELNET, sends text from the script file to the TELNET process, saves every character returned from the TELNET session, and “looks” for known character strings that the user “expected”. The script can test for different strings and execute different code based on the results. TELNET scripts can have a lot of intelligence built into them.

The first day at work after buying the book, I logged on to one of my HP/UNIX machines, did a man expect and received the message “no entry found”. After a few more tries, I made a call to HP. EXPECT and Tcl, the language EXPECT is based on, are not installed and are not even supplied on the CD-ROM distribution media from HP. HP recommended I download Tcl and EXPECT from a web site and install them. I do not like to do this sort of operation on a production server. After delivering a few favorite curse words, I went to my desktop Linux system.

When I installed Linux, I carved out a 1GB partition, did a full Linux install and dual booted it with MS Windows. A two-minute reboot, and Linux was up and running. Again, I ran man expect. This time, I got a man page—a very long and informative man page.

“Introduction EXPECT” is a program that “talks” to other interactive programs according to a script. Does this mean I have it? I ran the command

find / -name expect

and got this result:

/var/lib/LST/installed/expect
/var/lib/LST/contents/expect
/usr/bin/expect

Oh yeah, I've got it—I love Linux!

Poking around /usr/bin with the command ll | more, I found autoexpect. What's autoexpect? It's not in the book, so I typed man autoexpect and received the following output:

autoexpect - generate an EXPECT script from watching a session.

This is a cool program. I can run a TELNET or FTP session to a VAX from Linux, and create an EXPECT script from the session that I can run any time I wish.

Nothing beats actually doing it. I logged on to a VAX with FTP, put a file and quit. I copied the man page for EXPECT to a file, then I put it on a VAX using FTP.

man expect | col -b > man_expect.txt

Next, I used FTP to connect to a VAX named ZEUS and put the file man_expect.txt on that machine.

autoexpect -f test1.exp FTP ZEUS
autoexpect started, file is test1.exp
connected to ZEUS
220 Zeus FTP Server
Name (zeus:vinnie): v12321
Password:
230 User logged in.
Remote system type is VMS.
FTP> put man_expect.txt
 ...
FTP> quit
221 Goodbye.
Autoexpect done, file is test1.exp

The file test1.exp is the script created by autoexpect. I can rerun this script any time I wish simply by typing test1.exp at the prompt. Examining test1.exp shows the details of the two-way conversation between the Linux machine and the VAX. Every character is saved in either a send or expect command. Even the password is saved, so care must be taken with these scripts.

I edited my test1.exp script to eliminate the comments. I also took out all the non-essential expect commands, leaving only the bare essentials. The script was short and sweet. The whole concept of EXPECT is quite simple—send commands as normally typed, such as the VAX FTP prompt:

expect -exact "FTP> "

TELNET was easy too, but the script created by autoexpect would not run. Why? The basic problem of getting two computers to talk to each other is that they never say exactly the same thing twice; in particular, time and date character strings are always changing. Don points this out in the man pages for EXPECT, and again in his book.

When I used TELNET to log in to my VAX, the first thing it returned to me after the “Welcome” message was the time and date. After I did a “Directory Listing”, the file names and total number of files were returned. These responses vary with time. Removing them from the autoexpect scripts causes them to rerun perfectly every time. I wrote this article as an introduction to EXPECT, and to show how easy it is to use. Exploring EXPECT goes into detail on all the different aspects of EXPECT—from how it all works, to programming two-way conversations in EXPECT and how to log selected EXPECT output to a file. Many things you never thought could be automated can be run while you sit back and enjoy reading Linux Journal.