My Favorite bash Tips and Tricks
bash, or the Bourne again shell, is the default shell in most Linux distributions. The popularity of the bash shell amongst Linux and UNIX users is no accident. It has many features to enhance user-friendliness and productivity. Unfortunately, you can't take advantage of those features unless you know they exist.
When I first started using Linux, the only bash feature I took advantage of was going back through the command history using the up arrow. I soon learned additional features by watching others and asking questions. In this article, I'd like to share some bash tricks I've learned over the years.
This article isn't meant to cover all of the features of the bash shell; that would require a book, and plenty of books are available that cover this topic, including Learning the bash Shell from O'Reilly and Associates. Instead, this article is a summary of the bash tricks I use most often and would be lost without.
My favorite bash trick definitely is brace expansion. Brace expansion takes a list of strings separated by commas and expands those strings into separate arguments for you. The list is enclosed by braces, the symbols { and }, and there should be no spaces around the commas. For example:
$ echo {one,two,red,blue}
one two red blue
Using brace expansion as illustrated in this simple example doesn't offer too much to the user. In fact, the above example requires typing two more characters than simply typing:
echo one two red blue
which produces the same result. However, brace expansion becomes quite useful when the brace-enclosed list occurs immediately before, after or inside another string:
$ echo {one,two,red,blue}fish
onefish twofish redfish bluefish
$ echo fish{one,two,red,blue}
fishone fishtwo fishred fishblue
$ echo fi{one,two,red,blue}sh
fionesh fitwosh firedsh fibluesh
Notice that there are no spaces inside the brackets or between the brackets and the adjoining strings. If you include spaces, it breaks things:
$ echo {one, two, red, blue }fish
{one, two, red, blue }fish
$ echo "{one,two,red,blue} fish"
{one,two,red,blue} fish
However, you can use spaces if they're enclosed in quotes outside the braces or within an item in the comma-separated list:
$ echo {"one ","two ","red ","blue "}fish
one fish two fish red fish blue fish
$ echo {one,two,red,blue}" fish"
one fish two fish red fish blue fish
You also can nest braces, but you must use some caution here too:
$ echo {{1,2,3},1,2,3}
1 2 3 1 2 3
$ echo {{1,2,3}1,2,3}
11 21 31 2 3
Now, after all these examples, you might be thinking to yourself, “Gee, those are great parlor tricks, but why should I care about brace expansion?”
Brace expansion becomes useful when you need to make a backup of a file. This is why it's my favorite shell trick. I use it almost every day when I need to make a backup of a config file before changing it. For example, if I'm making a change to my Apache configuration, I can do the following and save some typing:
$ cp /etc/httpd/conf/httpd.conf{,.bak}
Notice that there is no character between the opening brace and the first comma. It's perfectly acceptable to do this and is useful when adding characters to an existing filename or when one argument is a substring of the other. Then, if I need to see what changes I made later in the day, I use the diff command and reverse the order of the strings inside the braces:
$ diff /etc/httpd/conf/httpd.conf{.bak,}
1050a1051
> # I added this comment earlier
Another bash trick I like to use is command substitution. To use command substitution, enclose any command that generates output to standard output inside parentheses and precede the opening parenthesis with a dollar sign, $(command). Command substitution is useful when assigning a value to a variable. This is typical in shell scripts, where a common operation is to assign the date or time to a variable. It also is handy for using the output of one command as an argument to another command. If you want to assign the date to a variable, you can do this:
$ date +%d-%b-%Y 12-Mar-2004 $ today=$(date +%d-%b-%Y) $ echo $today 12-Mar-2004
I often use command substitution to get information about several RPM packages at once. If I want a listing of all the files from all the RPM packages that have httpd in the name, I simply execute the following:
$ rpm -ql $(rpm -qa | grep httpd)
Today’s modular x86 servers are compute-centric, designed as a least common denominator to support a wide range of IT workloads. Those generic, virtualized IT workloads have much different resource optimization requirements than hyperscale and cloud applications. They have resulted in a “one size fits all” enterprise IT architecture that is not optimized for a specific set of IT workloads, and especially not emerging hyperscale workloads, such as web applications, big data, and object storage. In this report, you will learn how shifting the focus from traditional compute-centric IT architectures to an innovative disaggregated fabric-based architecture can optimize and scale your data center.
Sponsored by AMD
Built-in forensics, incident response, and security with Red Hat Enterprise Linux 6
Every security policy provides guidance and requirements for ensuring adequate protection of information and data, as well as high-level technical and administrative security requirements for a system in a given environment. Traditionally, providing security for a system focuses on the confidentiality of the information on it. However, protecting the data integrity and system and data availability is just as important. For example, when processing United States intelligence information, there are three attributes that require protection: confidentiality, integrity, and availability.
Learn more about catching the bad guy in this free white paper.
Sponsored by DLT Solutions
| Making Linux and Android Get Along (It's Not as Hard as It Sounds) | May 16, 2013 |
| Drupal Is a Framework: Why Everyone Needs to Understand This | May 15, 2013 |
| Home, My Backup Data Center | May 13, 2013 |
| Non-Linux FOSS: Seashore | May 10, 2013 |
| Trying to Tame the Tablet | May 08, 2013 |
| Dart: a New Web Programming Experience | May 07, 2013 |
- New Products
- Making Linux and Android Get Along (It's Not as Hard as It Sounds)
- A Topic for Discussion - Open Source Feature-Richness?
- Drupal Is a Framework: Why Everyone Needs to Understand This
- One Hand Slapping
- Home, My Backup Data Center
- What's the tweeting protocol?
- RSS Feeds
- Trying to Tame the Tablet
- Readers' Choice Awards 2011
- Reply to comment | Linux Journal
5 hours 1 min ago - Reply to comment | Linux Journal
7 hours 34 min ago - Reply to comment | Linux Journal
8 hours 51 min ago - great post
9 hours 26 min ago - Google Docs
9 hours 48 min ago - Reply to comment | Linux Journal
14 hours 37 min ago - Reply to comment | Linux Journal
15 hours 24 min ago - Web Hosting IQ
16 hours 57 min ago - Thanks for taking the time to
18 hours 34 min ago - Linux is good
20 hours 32 min ago
Enter to Win an Adafruit Prototyping Pi Plate Kit for Raspberry Pi

It's Raspberry Pi month at Linux Journal. Each week in May, Adafruit will be giving away a Pi-related prize to a lucky, randomly drawn LJ reader. Winners will be announced weekly.
Fill out the fields below to enter to win this week's prize-- a Prototyping Pi Plate Kit for Raspberry Pi.
Congratulations to our winners so far:
- 5-8-13, Pi Starter Pack: Jack Davis
- 5-15-13, Pi Model B 512MB RAM: Patrick Dunn
- Next winner announced on 5-21-13!
Free Webinar: Linux Backup and Recovery
Most companies incorporate backup procedures for critical data, which can be restored quickly if a loss occurs. However, fewer companies are prepared for catastrophic system failures, in which they lose all data, the entire operating system, applications, settings, patches and more, reducing their system(s) to “bare metal.” After all, before data can be restored to a system, there must be a system to restore it to.
In this one hour webinar, learn how to enhance your existing backup strategies for better disaster recovery preparedness using Storix System Backup Administrator (SBAdmin), a highly flexible bare-metal recovery solution for UNIX and Linux systems.




Comments
oldie but a goodie
Hey Prentice! Great article. Still doing any writing?
Hey Gavin, thanks! No, I
Hey Gavin, thanks! No, I haven't done any more writing in a while, but I should. I've been using a couple other bash "tricks" lately. Might have enough for a sequel.
eh?
instead of
to show files in a directory without invoking
any subprocesses, you could have just done
in fact, yours does't work properly if filenames
contain items such as asterixes or question marks
(amongst others)
try it
prints items twice, you need to surround your
echo with quotes
Reading the contents of a
Reading the contents of a file FILE using bash builtins:
while line;do echo $line;done
Another example of brace expansion
I needed to create directories to hold some batch files on Windows.
Right clicking and creating "batch1", "batch2" etc got boring once I got to "batch3".
I simply opened up my cygwin shell and did:
mkdir batch{1,2,3,4,5,6,7,8}
:)
brace expansions iterate too
instead of batch{1,2,3,4} etc try this batch{1..100}
win.
Inquire on sorting field in file
Hi,
Thanks for the article. However, i am trying to solve the below problem:-
Given the file content is:-
Count Line
0 1
1 1
2 10
3 1
2 3
If I would like to map each line with the count and also add the count for each line, what command should I used? I tried to have a for loop with cut for firled in the file but I do notknow how to extract these array according to line no.
Please advise. Thanks
-Lee
This is help full for me,
This is help full for me, thank. Some other bash commands can be found here : http://24hnews.net/General/25-Linux-UNIX-commands-that-you-must-know.html
> Now, can anyone suggest a
> Now, can anyone suggest a way to display the contents of a file using
> only bash built-ins?
# source /path/to/filename
Quote: Can anyone tell me
Quote:
Can anyone tell me how to list the contents of a file using only the bash built-ins?
My answer:
Take advantage of the catalog command and pipe it into awk that parses for (in this example) double quotes and returns the zeroth result, hence printing the entire file.
cat /filename | awk -F "\"" '{ print $0 }'
VIOLA, file contents printed.
Use a variant of this command on my webserver to parse logs for the names of webbots to keep the d@$n things from eating all my band.
HellMINTH aka christopmj
oh it is tempting to leave...
oh it is tempting to leave the solution of only using builtins off the replies as it got eaten by the server and its hunger for <
But here it is:
exec 5<file
while read -u5 k; do echo $k; done
nice trick but why not while
nice trick but why not while read a; do echo "$a"; done < file ?
remember to put "" around your var, or you'll have bash parsing the contents of $k and destroying spaces
cat is not actually the
cat is not actually the catalog/catalogue command, but in fact the concatenate command; man cat will show you this.
So, really cat's main purpose is to join two or more files together.
To output a file (list the contents of a file) using bash built-ins:
Bash out-of-meory hacks
The for loop technique for listing a directory is nice, but this is
fewer keystrokes and shows more files if you can't pipe to more:
echo *
To display the contents of a file, the simplest command I see is
echo $(<file)
According to strace(1), the shell clones itself, opens and reads
the file in the subprocess, and then the main process writes the
file's contents to stdout. This could fail for lack of a process
table slot or if it needed more memory and brk(2) failed, but it
doesn't exec anything else.
Paul Allen
another 'classic'
One of my personal favorites: turn any command into a (sort of) full
screen monitor, e.g. :
while [ 1 ]; do clear; w; sleep 3; done
Very simplistic (doesn't check for output > 25/24, etc) but good
enough for common purposes.
I also like subshells a lot :
(cd /somewhere && do something) | \
(cd /somewhere/else && read from something)
Maybe a 'My Favorite bash Tips and Tricks - Part II' is in order?
instead of "while [ 1 ]; do
instead of "while [ 1 ]; do clear; w; sleep 3; done"
"watch -n 3 w" is preferable if you want to watch the output of a command over time
yep but watch won't work for
yep but watch won't work for more complex commands, for the example that was written, monitoring 'w' it is okay, but for something with pipes or redirections in it it will not work!
It works just fine, all you
It works just fine, all you have to do is escaping the special characters. For example:
watch df -h \; echo \; dmesg \| tail
This prints the output of df, a blank line and the last lines of dmesg. You can also use && and || (which has to be written as \&\& or '&&').
or put the whole thing into
or put the whole thing into double quotes, like:
watch "ls -x ; df ."
but one disadvantage with "watch" is, that it strips output off coloring codes etc.
so if you wanna observe output which contains any hilighting, you will end up with plain uniform text, sadly. And as far as i know, the while ... sleep construct is the only usable workaround here. Sometimes i also add "date" as first command, just to be sure when the snapshot was taken :)
Hint me if you find any better approach :)
But there's also some major advantage using watch:
-d parameter automatically diffs the output to the previous
And screen formatting is automatically done.
Plus it shows the timestamp with each output.
Emacs?
Thanks for a very helpful and useful article. One thing I would suggest adding is that the
Ctrl-Rtrick is the Emacs shortcut for reverse incremental search. And in general that many Emacs shortcuts (Ctrl-A, Ctrl-E, Alt-d, Alt-<DEL>) can be used to edit commands at Bash prompt.Display file contents using bash built-ins
To display the contents of a file (file.txt in this case) using BASH built-ins you could use:
while read line; do echo "$line"; done <file.txt
Rick Malone
Systems Engineer Technician
Simulation Training Centre
Petawawa, ON Canada
more as a script
Here is the result of a discussion I had about how to implement more using just a shell script (almost ten years ago :-). The goal was just to have fun, but the simplistic result is handy when the terminal (such as a Sparc10 console) doesn't have a scrollback buffer. Sadly I haven't been able to find the other person involved, but here ya go (note: the indentation doesn't show up):
shmore() {
LINES=""
while read line
do
echo "$line"
LINES=".$LINES"
if [ $LINES == "......................." ]; then
echo -n "--More--"
read < /dev/tty
LINES=""
fi
done
}
Now you can load it up,
. more.sh, and use it:shmore < somefileSteve Stock
cat.sh
---cut---
#!/bin/sh
if [ $# = 0 ]
then
while read -r; do echo $REPLY; done
else
for F in "$@"
do
while read -r; do echo $REPLY; done < "$F"
done
fi
---cut---