Automating Tasks with EXPECT
As the System Administrator for 18 VAXs and 6 HP/UNIX machines, I am always looking for an easier way to do things. One thing I always do is write shell scripts for UNIX and command procedures for VMS to automate some of my tasks. If I could run them from one computer, I could administer the world from one place without logging on to each computer to run each task I have to perform.
I first tried using VAX/VMS command files in order to TELNET to the other machines. That method was quickly dropped, as the remote computer's I/O response arrived before my command file could do an input statement. Logons hung up quickly. I then tried to TELNET from a UNIX script—the same thing occurred. This type of frustration is why an administrator's job is tough.
Hanging out in a Barnes & Noble bookstore and leaning on the UNIX shelves is always a good pastime—you look good, and you help dust off the books. Leaning is how I discovered Exploring EXPECT, an interesting title for a book with a back cover stating “automate TELNET, FTP, passwd...”. I bought the book—yes, I judge a book by its cover (sometimes).
The excellent book, Exploring EXPECT by Don Libes (published by O'Reilly & Associates) contains everything you need to know about controlling any spawned process. You can spawn a game and interact with it as easily as a remote TELNET session. The book is fairly long and very detailed, but don't be intimidated. There is an easy way to learn to write EXPECT scripts; I'll get to it shortly.
To make a long story short, EXPECT is a toolkit for automating interactive programs, such as TELNET and FTP, written by Don Libes. EXPECT has to be the greatest asset to system administrators in all of history. Finally, I can easily write and execute TELNET logon scripts and do whatever I wish using a script file from one machine.
EXPECT reads commands from a script file, spawns a process like TELNET, sends text from the script file to the TELNET process, saves every character returned from the TELNET session, and “looks” for known character strings that the user “expected”. The script can test for different strings and execute different code based on the results. TELNET scripts can have a lot of intelligence built into them.
The first day at work after buying the book, I logged on to one of my HP/UNIX machines, did a man expect and received the message “no entry found”. After a few more tries, I made a call to HP. EXPECT and Tcl, the language EXPECT is based on, are not installed and are not even supplied on the CD-ROM distribution media from HP. HP recommended I download Tcl and EXPECT from a web site and install them. I do not like to do this sort of operation on a production server. After delivering a few favorite curse words, I went to my desktop Linux system.
When I installed Linux, I carved out a 1GB partition, did a full Linux install and dual booted it with MS Windows. A two-minute reboot, and Linux was up and running. Again, I ran man expect. This time, I got a man page—a very long and informative man page.
“Introduction EXPECT” is a program that “talks” to other interactive programs according to a script. Does this mean I have it? I ran the command
find / -name expect
and got this result:
/var/lib/LST/installed/expect /var/lib/LST/contents/expect /usr/bin/expectOh yeah, I've got it—I love Linux!
Poking around /usr/bin with the command ll | more, I found autoexpect. What's autoexpect? It's not in the book, so I typed man autoexpect and received the following output:
autoexpect - generate an EXPECT script from watching a session.
This is a cool program. I can run a TELNET or FTP session to a VAX from Linux, and create an EXPECT script from the session that I can run any time I wish.
Nothing beats actually doing it. I logged on to a VAX with FTP, put a file and quit. I copied the man page for EXPECT to a file, then I put it on a VAX using FTP.
man expect | col -b > man_expect.txt
Next, I used FTP to connect to a VAX named ZEUS and put the file man_expect.txt on that machine.
autoexpect -f test1.exp FTP ZEUS autoexpect started, file is test1.exp connected to ZEUS 220 Zeus FTP Server Name (zeus:vinnie): v12321 Password: 230 User logged in. Remote system type is VMS. FTP> put man_expect.txt ... FTP> quit 221 Goodbye. Autoexpect done, file is test1.expThe file test1.exp is the script created by autoexpect. I can rerun this script any time I wish simply by typing test1.exp at the prompt. Examining test1.exp shows the details of the two-way conversation between the Linux machine and the VAX. Every character is saved in either a send or expect command. Even the password is saved, so care must be taken with these scripts.
I edited my test1.exp script to eliminate the comments. I also took out all the non-essential expect commands, leaving only the bare essentials. The script was short and sweet. The whole concept of EXPECT is quite simple—send commands as normally typed, such as the VAX FTP prompt:
expect -exact "FTP> "
TELNET was easy too, but the script created by autoexpect would not run. Why? The basic problem of getting two computers to talk to each other is that they never say exactly the same thing twice; in particular, time and date character strings are always changing. Don points this out in the man pages for EXPECT, and again in his book.
When I used TELNET to log in to my VAX, the first thing it returned to me after the “Welcome” message was the time and date. After I did a “Directory Listing”, the file names and total number of files were returned. These responses vary with time. Removing them from the autoexpect scripts causes them to rerun perfectly every time. I wrote this article as an introduction to EXPECT, and to show how easy it is to use. Exploring EXPECT goes into detail on all the different aspects of EXPECT—from how it all works, to programming two-way conversations in EXPECT and how to log selected EXPECT output to a file. Many things you never thought could be automated can be run while you sit back and enjoy reading Linux Journal.
Vinnie Saladino has been immersed in UNIX for 4 years, and VAX for 19 years. He has a BS in Electrical Engineering and an MS in Computer Science. Currently, UNIX and Oracle take up most of his time. He enjoys building, nailing, sawing and wiring, as well as working with stained glass. He lives with his two kids, two dogs and two computers. He welcomes your comments at email@example.com.
Practical Task Scheduling Deployment
July 20, 2016 12:00 pm CDT
One of the best things about the UNIX environment (aside from being stable and efficient) is the vast array of software tools available to help you do your job. Traditionally, a UNIX tool does only one thing, but does that one thing very well. For example, grep is very easy to use and can search vast amounts of data quickly. The find tool can find a particular file or files based on all kinds of criteria. It's pretty easy to string these tools together to build even more powerful tools, such as a tool that finds all of the .log files in the /home directory and searches each one for a particular entry. This erector-set mentality allows UNIX system administrators to seem to always have the right tool for the job.
Cron traditionally has been considered another such a tool for job scheduling, but is it enough? This webinar considers that very question. The first part builds on a previous Geek Guide, Beyond Cron, and briefly describes how to know when it might be time to consider upgrading your job scheduling infrastructure. The second part presents an actual planning and implementation framework.
Join Linux Journal's Mike Diehl and Pat Cameron of Help Systems.
Free to Linux Journal readers.Register Now!
- SUSE LLC's SUSE Manager
- Murat Yener and Onur Dundar's Expert Android Studio (Wrox)
- My +1 Sword of Productivity
- Managing Linux Using Puppet
- Non-Linux FOSS: Caffeine!
- Doing for User Space What We Did for Kernel Space
- Google's SwiftShader Released
- SuperTuxKart 0.9.2 Released
- LiveCode Ltd.'s LiveCode
- Parsing an RSS News Feed with a Bash Script
With all the industry talk about the benefits of Linux on Power and all the performance advantages offered by its open architecture, you may be considering a move in that direction. If you are thinking about analytics, big data and cloud computing, you would be right to evaluate Power. The idea of using commodity x86 hardware and replacing it every three years is an outdated cost model. It doesn’t consider the total cost of ownership, and it doesn’t consider the advantage of real processing power, high-availability and multithreading like a demon.
This ebook takes a look at some of the practical applications of the Linux on Power platform and ways you might bring all the performance power of this open architecture to bear for your organization. There are no smoke and mirrors here—just hard, cold, empirical evidence provided by independent sources. I also consider some innovative ways Linux on Power will be used in the future.Get the Guide