Automating Tasks with EXPECT

Mr. Saladino gives a quick introduction to EXPECT, a program to help you accomplish your remote tasks.

As the System Administrator for 18 VAXs and 6 HP/UNIX machines, I am always looking for an easier way to do things. One thing I always do is write shell scripts for UNIX and command procedures for VMS to automate some of my tasks. If I could run them from one computer, I could administer the world from one place without logging on to each computer to run each task I have to perform.

I first tried using VAX/VMS command files in order to TELNET to the other machines. That method was quickly dropped, as the remote computer's I/O response arrived before my command file could do an input statement. Logons hung up quickly. I then tried to TELNET from a UNIX script—the same thing occurred. This type of frustration is why an administrator's job is tough.

Hanging out in a Barnes & Noble bookstore and leaning on the UNIX shelves is always a good pastime—you look good, and you help dust off the books. Leaning is how I discovered Exploring EXPECT, an interesting title for a book with a back cover stating “automate TELNET, FTP, passwd...”. I bought the book—yes, I judge a book by its cover (sometimes).

The excellent book, Exploring EXPECT by Don Libes (published by O'Reilly & Associates) contains everything you need to know about controlling any spawned process. You can spawn a game and interact with it as easily as a remote TELNET session. The book is fairly long and very detailed, but don't be intimidated. There is an easy way to learn to write EXPECT scripts; I'll get to it shortly.

To make a long story short, EXPECT is a toolkit for automating interactive programs, such as TELNET and FTP, written by Don Libes. EXPECT has to be the greatest asset to system administrators in all of history. Finally, I can easily write and execute TELNET logon scripts and do whatever I wish using a script file from one machine.

EXPECT reads commands from a script file, spawns a process like TELNET, sends text from the script file to the TELNET process, saves every character returned from the TELNET session, and “looks” for known character strings that the user “expected”. The script can test for different strings and execute different code based on the results. TELNET scripts can have a lot of intelligence built into them.

The first day at work after buying the book, I logged on to one of my HP/UNIX machines, did a man expect and received the message “no entry found”. After a few more tries, I made a call to HP. EXPECT and Tcl, the language EXPECT is based on, are not installed and are not even supplied on the CD-ROM distribution media from HP. HP recommended I download Tcl and EXPECT from a web site and install them. I do not like to do this sort of operation on a production server. After delivering a few favorite curse words, I went to my desktop Linux system.

When I installed Linux, I carved out a 1GB partition, did a full Linux install and dual booted it with MS Windows. A two-minute reboot, and Linux was up and running. Again, I ran man expect. This time, I got a man page—a very long and informative man page.

“Introduction EXPECT” is a program that “talks” to other interactive programs according to a script. Does this mean I have it? I ran the command

find / -name expect

and got this result:

/var/lib/LST/installed/expect
/var/lib/LST/contents/expect
/usr/bin/expect
Oh yeah, I've got it—I love Linux!

Poking around /usr/bin with the command ll | more, I found autoexpect. What's autoexpect? It's not in the book, so I typed man autoexpect and received the following output:

autoexpect - generate an EXPECT script from watching a session.

This is a cool program. I can run a TELNET or FTP session to a VAX from Linux, and create an EXPECT script from the session that I can run any time I wish.

Nothing beats actually doing it. I logged on to a VAX with FTP, put a file and quit. I copied the man page for EXPECT to a file, then I put it on a VAX using FTP.

man expect | col -b > man_expect.txt

Next, I used FTP to connect to a VAX named ZEUS and put the file man_expect.txt on that machine.

autoexpect -f test1.exp FTP ZEUS
autoexpect started, file is test1.exp
connected to ZEUS
220 Zeus FTP Server
Name (zeus:vinnie): v12321
Password:
230 User logged in.
Remote system type is VMS.
FTP> put man_expect.txt
 ...
FTP> quit
221 Goodbye.
Autoexpect done, file is test1.exp
The file test1.exp is the script created by autoexpect. I can rerun this script any time I wish simply by typing test1.exp at the prompt. Examining test1.exp shows the details of the two-way conversation between the Linux machine and the VAX. Every character is saved in either a send or expect command. Even the password is saved, so care must be taken with these scripts.

I edited my test1.exp script to eliminate the comments. I also took out all the non-essential expect commands, leaving only the bare essentials. The script was short and sweet. The whole concept of EXPECT is quite simple—send commands as normally typed, such as the VAX FTP prompt:

expect -exact "FTP> "

TELNET was easy too, but the script created by autoexpect would not run. Why? The basic problem of getting two computers to talk to each other is that they never say exactly the same thing twice; in particular, time and date character strings are always changing. Don points this out in the man pages for EXPECT, and again in his book.

When I used TELNET to log in to my VAX, the first thing it returned to me after the “Welcome” message was the time and date. After I did a “Directory Listing”, the file names and total number of files were returned. These responses vary with time. Removing them from the autoexpect scripts causes them to rerun perfectly every time. I wrote this article as an introduction to EXPECT, and to show how easy it is to use. Exploring EXPECT goes into detail on all the different aspects of EXPECT—from how it all works, to programming two-way conversations in EXPECT and how to log selected EXPECT output to a file. Many things you never thought could be automated can be run while you sit back and enjoy reading Linux Journal.

Vinnie Saladino has been immersed in UNIX for 4 years, and VAX for 19 years. He has a BS in Electrical Engineering and an MS in Computer Science. Currently, UNIX and Oracle take up most of his time. He enjoys building, nailing, sawing and wiring, as well as working with stained glass. He lives with his two kids, two dogs and two computers. He welcomes your comments at saladino@idt.net.

______________________

Comments

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

interact hangs out when I cat a large file

Anonymous's picture

Hi,

I would like to know whether there is any fix for to prevent interact command from hanging out when I cat a large inside a expect spawned process. Send in your reply to gururaj.srinivasarao@gmail.com

I found an amazing tutorial

Anonymous's picture

you can check out this amazing tutorial its the best
http://exploring-expect-tcl.blog.com/2009/06/03/exploring-expect/

Very Good knowledge shaired

Shailesh's picture

Thanks.
This is very good for system admins.

How to write regular expression for welcome message?

Anonymous's picture

After login welcome message shows date and time .. How do we write RE for it ? Any idea.

TIA

Thanks !!! It helped me alot

Anonymous's picture

Thanks !!! It helped me alot !!!!!

Expect buffer

Anonymous's picture

Hello,

I want to know the size of expect_out(buffer).I want large amount of data to be moved to file using expect buffer but match_max did not help.

Can you please suggest me solution?

Thanks,
pankaj

Telnet session on a telnet session through EXPECT

Sapnesh's picture

Hi All,

I want to telnet to server A followed by another telnet to Server B from Server A.

I could create the expect scripts to telnet to a single server. But then if i send command 'send "telnet server B\r"`to Server A, it doesnot respond properly.

Please Help. do i need to spawn the second telnet process? If yes, then how would i pass the spawn id information back to local machine.

ThanksInAdvance,
Sapnesh

Your guide is very helpfull

rozi's picture

Hi,
I'm seeking the guide that can help me solveing the auto task rather than crond. Now i found it. Great help!

Great solution

Chuck D Smith's picture

This article gave us a great hint to creating a solution to rebooting a Perle Specialix Jetstream terminal server for a failover test. The 'autoexpect' utility gave us the solution we needed in 5 minutes after we found the article. Great stuff! And the article is 8 years old at that!
Please renew my subscription... Oh yeah I just did renew it...

Great help! Thanks!

Anonymous's picture

Great help! Thanks!

Re: System Administration: Automating Tasks with EXPECT

Anonymous's picture

It was a great info I was earlier looking for!
Long live :-)
Kumar

Re: System Administration: Automating Tasks with EXPECT

Anonymous's picture

How to automate telnet through this autoexpect.....can someone help me out....

thanks in advance

S.Sriram

Automating Telnet with expect

Yash's picture

Hi Sriram,

This is too easy.

Just type

$ autoexpect -f telnet.exp telnet

and then enter your username and password
execute any command if u want on remote machine. Then finally exit.

After this telnet.exp file will be created.
Open telnet.exp file and delete the lines containing time stamp for your login. Save the changes. And run ./telnet.exp.

Hope this helps :)
Yash

Re: System Administration: Automating Tasks with EXPECT

Anonymous's picture

this acticle is too sample to help someone understand the Expect

Re: System Administration: Automating Tasks with EXPECT

Anonymous's picture

you just have to do :

autoexpect telnet X.X.X.X

then do like you were doing a standart telnet, enter all the command you need.

then, quit the telnet and a script will be generated in your folder.

juste lunch the script to see what you do before.

I hope it will help you.

Phil.

I was just wondering if

Anonymous's picture

I was just wondering if anyone knew how to make the ip address you are telnet into a variable. I am using a script that was created by the autoexpect feature, but i want to be able to choose the ip address i am going to telnet. thanks for your help.

Telnet - variable IP

George Shelton's picture

Here you go, Host name or IP will work.

# Get the Host name that you are going to work on

send_user "Input Host name here: "

expect_user -re "(.*)\n" {set HostName $expect_out(1,string) }

send_user "\n"
spawn telnet $HostName

Webinar
One Click, Universal Protection: Implementing Centralized Security Policies on Linux Systems

As Linux continues to play an ever increasing role in corporate data centers and institutions, ensuring the integrity and protection of these systems must be a priority. With 60% of the world's websites and an increasing share of organization's mission-critical workloads running on Linux, failing to stop malware and other advanced threats on Linux can increasingly impact an organization's reputation and bottom line.

Learn More

Sponsored by Bit9

Webinar
Linux Backup and Recovery Webinar

Most companies incorporate backup procedures for critical data, which can be restored quickly if a loss occurs. However, fewer companies are prepared for catastrophic system failures, in which they lose all data, the entire operating system, applications, settings, patches and more, reducing their system(s) to “bare metal.” After all, before data can be restored to a system, there must be a system to restore it to.

In this one hour webinar, learn how to enhance your existing backup strategies for better disaster recovery preparedness using Storix System Backup Administrator (SBAdmin), a highly flexible bare-metal recovery solution for UNIX and Linux systems.

Learn More

Sponsored by Storix