Bourne Shell Scripts
At some point in your dealings with UNIX and Linux, you may need to write a script which edits critical system files such as /etc/passwd or the rc boot scripts. Whatever the file, if your script messes it up, your machine will suffer because of it. Half-hearted scripting, which may be sufficient for normal files, will not work in these situations.
Consider the root password. If you need to modify the root password on one machine, it's quite easy using the passwd command. But if you need to change the root password on 50 or more machines, the problem is much more difficult. It could still be done by hand, but this would be time-consuming and prone to errors. One way to solve this problem is by using EX.
If you view the man page for EX on Red Hat Linux, you will get the man page for vim instead. This is no mistake. EX is the engine underneath the familiar vi interface. All the power of vi is available from within your scripts, if you know how to access it.
Let's start with some simple edits and work our way up to the root password change example.
Listing 1 shows a simple multi-line file. Suppose we wanted to remove the first line containing the word “three”. We could write the short script in Listing 2 to accomplish this easily.
Lines 1 through 6 in Listing 2 should be familiar to anyone who has done shell scripting. If you have never written a script, I recommend the book Learning the bash Shell (see Resources) to get started.
Line 5 is where the interesting stuff begins. We call EX with the -s option to tell EX to run in silent mode. The file we are operating on is entered next, in the $TARGET variable. The uninteresting output of EX is redirected to /dev/null because we don't want it to clutter any output of the script itself. You can delete >/dev/null if you would like to see this information during debug. The <<E_O_F marks the beginning of a “Here” file.
A Here file is a way of placing the contents of a file within a script. The line after <<E_O_F is the first line of the file. The Here file consists of all lines up to the line beginning with E_O_F. This tag (which ends the Here file) can be any unique character combination. We are using E_O_F here out of convention. The “<<” has the effect of feeding the contents of the Here file to EX, which uses those commands to edit the $TARGET file. Notice that the Here file is not indented. If we were to change <<E_O_F to <<-E_O_F, adding the “-”, we could indent the Here file with TAB characters. If those TAB characters somehow became spaces, perhaps by a copy-and-paste operation, the Here file will cease to work, the E_O_F will never be seen and the contents of your script below the E_O_F marker will be fed to EX, creating unpredictable output. For this reason, I make it a personal policy to never indent Here files. For more information on Here files, see the book UNIX Power Tools (in Resources).
The first line of the Here file is line 6, which contains a 1. This forces the line pointer in EX to the first line of the file. While this is not completely necessary for the script to work, it does eliminate any confusion as to where subsequent searches begin during your edit. I find it to be a helpful programming practice.
Line 7 is a regular expression. EX uses this regular expression to search through the $TARGET file and set its line pointer to the first line containing the word “three”. In listing 1, this is line 3. The trailing “d” in line 10's regular expression tells EX to delete that line. Line 8 is familiar to vi users. It says to write, quit and “don't argue with me”. Line 9 is the E_O_F tag, which marks the end of the Here file.
We could add lines to the file with the a or i commands. Listing 3 shows examples of this. Notice the trailing “d” is gone from the regular expression. We are using the regular expression to set the line pointer, not delete the line. The a appends a line after the line defined in the regular expression, and the i command inserts a line before the line defined in the regular expression. The . command tells EX to leave edit mode, similar to pressing the ESC key in vi. The last EX example in Listing 3 changes the 1 to a $, forcing the line pointer to the end of the file. We then append a line, which has the same effect as this command:
echo "This line is added at the bottom of the \ file." >> $TARGET
Now that we have simple edits under our belt, let's look at the more complicated problem mentioned earlier of changing multiple root passwords. Once we have changed one password with the passwd command, we can copy and paste the password hash from /etc/shadow into a variable in a script. Listing 4 contains the script. The REPL_STRING variable contains the new password hash. Notice we had to escape “$”, “.” and “/” with backslashes to prevent EX from interpreting these as special characters and giving unpredictable output. As you see, the only major difference between this script and the earlier examples is the regular expression.
Due to space constraints, I will not fully explain the regular expression here except to say it has the effect of “remembering” everything on each side of the existing password field and adding that to each side of the REPL_STRING variable, making up a new root line in the /etc/passwd file. At the end of the script, we use pwconv to put the new password hash into the /etc/shadow file. This script can be put in a place that all machines can mount and can be remotely executed by each machine, giving them all the same root password.
As you can see, the file editing power of your scripts with EX is limited only by your knowledge of regular expressions. If you don't currently use regular expressions, they are well worth taking the time to learn. For more information on regular expressions, read the book Mastering Regular Expressions (see Resources).
Practical Task Scheduling Deployment
July 20, 2016 12:00 pm CDT
One of the best things about the UNIX environment (aside from being stable and efficient) is the vast array of software tools available to help you do your job. Traditionally, a UNIX tool does only one thing, but does that one thing very well. For example, grep is very easy to use and can search vast amounts of data quickly. The find tool can find a particular file or files based on all kinds of criteria. It's pretty easy to string these tools together to build even more powerful tools, such as a tool that finds all of the .log files in the /home directory and searches each one for a particular entry. This erector-set mentality allows UNIX system administrators to seem to always have the right tool for the job.
Cron traditionally has been considered another such a tool for job scheduling, but is it enough? This webinar considers that very question. The first part builds on a previous Geek Guide, Beyond Cron, and briefly describes how to know when it might be time to consider upgrading your job scheduling infrastructure. The second part presents an actual planning and implementation framework.
Join Linux Journal's Mike Diehl and Pat Cameron of Help Systems.
Free to Linux Journal readers.Register Now!
- SUSE LLC's SUSE Manager
- My +1 Sword of Productivity
- Murat Yener and Onur Dundar's Expert Android Studio (Wrox)
- Managing Linux Using Puppet
- Non-Linux FOSS: Caffeine!
- Doing for User Space What We Did for Kernel Space
- SuperTuxKart 0.9.2 Released
- Google's SwiftShader Released
- Parsing an RSS News Feed with a Bash Script
- Rogue Wave Software's Zend Server
With all the industry talk about the benefits of Linux on Power and all the performance advantages offered by its open architecture, you may be considering a move in that direction. If you are thinking about analytics, big data and cloud computing, you would be right to evaluate Power. The idea of using commodity x86 hardware and replacing it every three years is an outdated cost model. It doesn’t consider the total cost of ownership, and it doesn’t consider the advantage of real processing power, high-availability and multithreading like a demon.
This ebook takes a look at some of the practical applications of the Linux on Power platform and ways you might bring all the performance power of this open architecture to bear for your organization. There are no smoke and mirrors here—just hard, cold, empirical evidence provided by independent sources. I also consider some innovative ways Linux on Power will be used in the future.Get the Guide