Bourne Shell Scripts
At some point in your dealings with UNIX and Linux, you may need to write a script which edits critical system files such as /etc/passwd or the rc boot scripts. Whatever the file, if your script messes it up, your machine will suffer because of it. Half-hearted scripting, which may be sufficient for normal files, will not work in these situations.
Consider the root password. If you need to modify the root password on one machine, it's quite easy using the passwd command. But if you need to change the root password on 50 or more machines, the problem is much more difficult. It could still be done by hand, but this would be time-consuming and prone to errors. One way to solve this problem is by using EX.
If you view the man page for EX on Red Hat Linux, you will get the man page for vim instead. This is no mistake. EX is the engine underneath the familiar vi interface. All the power of vi is available from within your scripts, if you know how to access it.
Let's start with some simple edits and work our way up to the root password change example.
Listing 1 shows a simple multi-line file. Suppose we wanted to remove the first line containing the word “three”. We could write the short script in Listing 2 to accomplish this easily.
Lines 1 through 6 in Listing 2 should be familiar to anyone who has done shell scripting. If you have never written a script, I recommend the book Learning the bash Shell (see Resources) to get started.
Line 5 is where the interesting stuff begins. We call EX with the -s option to tell EX to run in silent mode. The file we are operating on is entered next, in the $TARGET variable. The uninteresting output of EX is redirected to /dev/null because we don't want it to clutter any output of the script itself. You can delete >/dev/null if you would like to see this information during debug. The <<E_O_F marks the beginning of a “Here” file.
A Here file is a way of placing the contents of a file within a script. The line after <<E_O_F is the first line of the file. The Here file consists of all lines up to the line beginning with E_O_F. This tag (which ends the Here file) can be any unique character combination. We are using E_O_F here out of convention. The “<<” has the effect of feeding the contents of the Here file to EX, which uses those commands to edit the $TARGET file. Notice that the Here file is not indented. If we were to change <<E_O_F to <<-E_O_F, adding the “-”, we could indent the Here file with TAB characters. If those TAB characters somehow became spaces, perhaps by a copy-and-paste operation, the Here file will cease to work, the E_O_F will never be seen and the contents of your script below the E_O_F marker will be fed to EX, creating unpredictable output. For this reason, I make it a personal policy to never indent Here files. For more information on Here files, see the book UNIX Power Tools (in Resources).
The first line of the Here file is line 6, which contains a 1. This forces the line pointer in EX to the first line of the file. While this is not completely necessary for the script to work, it does eliminate any confusion as to where subsequent searches begin during your edit. I find it to be a helpful programming practice.
Line 7 is a regular expression. EX uses this regular expression to search through the $TARGET file and set its line pointer to the first line containing the word “three”. In listing 1, this is line 3. The trailing “d” in line 10's regular expression tells EX to delete that line. Line 8 is familiar to vi users. It says to write, quit and “don't argue with me”. Line 9 is the E_O_F tag, which marks the end of the Here file.
We could add lines to the file with the a or i commands. Listing 3 shows examples of this. Notice the trailing “d” is gone from the regular expression. We are using the regular expression to set the line pointer, not delete the line. The a appends a line after the line defined in the regular expression, and the i command inserts a line before the line defined in the regular expression. The . command tells EX to leave edit mode, similar to pressing the ESC key in vi. The last EX example in Listing 3 changes the 1 to a $, forcing the line pointer to the end of the file. We then append a line, which has the same effect as this command:
echo "This line is added at the bottom of the \ file." >> $TARGET
Now that we have simple edits under our belt, let's look at the more complicated problem mentioned earlier of changing multiple root passwords. Once we have changed one password with the passwd command, we can copy and paste the password hash from /etc/shadow into a variable in a script. Listing 4 contains the script. The REPL_STRING variable contains the new password hash. Notice we had to escape “$”, “.” and “/” with backslashes to prevent EX from interpreting these as special characters and giving unpredictable output. As you see, the only major difference between this script and the earlier examples is the regular expression.
Due to space constraints, I will not fully explain the regular expression here except to say it has the effect of “remembering” everything on each side of the existing password field and adding that to each side of the REPL_STRING variable, making up a new root line in the /etc/passwd file. At the end of the script, we use pwconv to put the new password hash into the /etc/shadow file. This script can be put in a place that all machines can mount and can be remotely executed by each machine, giving them all the same root password.
As you can see, the file editing power of your scripts with EX is limited only by your knowledge of regular expressions. If you don't currently use regular expressions, they are well worth taking the time to learn. For more information on regular expressions, read the book Mastering Regular Expressions (see Resources).
|Using Salt Stack and Vagrant for Drupal Development||May 20, 2013|
|Making Linux and Android Get Along (It's Not as Hard as It Sounds)||May 16, 2013|
|Drupal Is a Framework: Why Everyone Needs to Understand This||May 15, 2013|
|Home, My Backup Data Center||May 13, 2013|
|Non-Linux FOSS: Seashore||May 10, 2013|
|Trying to Tame the Tablet||May 08, 2013|
- RSS Feeds
- Making Linux and Android Get Along (It's Not as Hard as It Sounds)
- Using Salt Stack and Vagrant for Drupal Development
- New Products
- Validate an E-Mail Address with PHP, the Right Way
- Drupal Is a Framework: Why Everyone Needs to Understand This
- A Topic for Discussion - Open Source Feature-Richness?
- Download the Free Red Hat White Paper "Using an Open Source Framework to Catch the Bad Guy"
- Tech Tip: Really Simple HTTP Server with Python
- Readers' Choice Awards
- Please correct the URL for Salt Stack's web site
4 min 7 sec ago
- Android is Linux -- why no better inter-operation
2 hours 19 min ago
- Connecting Android device to desktop Linux via USB
2 hours 47 min ago
- Find new cell phone and tablet pc
3 hours 46 min ago
5 hours 14 min ago
- Automatically updating Guest Additions
6 hours 23 min ago
- I like your topic on android
7 hours 9 min ago
- Reply to comment | Linux Journal
7 hours 31 min ago
- This is the easiest tutorial
13 hours 45 min ago
- Ahh, the Koolaid.
19 hours 24 min ago
Enter to Win an Adafruit Pi Cobbler Breakout Kit for Raspberry Pi
It's Raspberry Pi month at Linux Journal. Each week in May, Adafruit will be giving away a Pi-related prize to a lucky, randomly drawn LJ reader. Winners will be announced weekly.
Fill out the fields below to enter to win this week's prize-- a Pi Cobbler Breakout Kit for Raspberry Pi.
Congratulations to our winners so far:
- 5-8-13, Pi Starter Pack: Jack Davis
- 5-15-13, Pi Model B 512MB RAM: Patrick Dunn
- 5-21-13, Prototyping Pi Plate Kit: Philip Kirby
- Next winner announced on 5-27-13!
Free Webinar: Hadoop
How to Build an Optimal Hadoop Cluster to Store and Maintain Unlimited Amounts of Data Using Microservers
Realizing the promise of Apache® Hadoop® requires the effective deployment of compute, memory, storage and networking to achieve optimal results. With its flexibility and multitude of options, it is easy to over or under provision the server infrastructure, resulting in poor performance and high TCO. Join us for an in depth, technical discussion with industry experts from leading Hadoop and server companies who will provide insights into the key considerations for designing and deploying an optimal Hadoop cluster.
Some of key questions to be discussed are:
- What is the “typical” Hadoop cluster and what should be installed on the different machine types?
- Why should you consider the typical workload patterns when making your hardware decisions?
- Are all microservers created equal for Hadoop deployments?
- How do I plan for expansion if I require more compute, memory, storage or networking?