Bash Redirections Using Exec
If you've used the command line much at all you know about I/O redirection for redirecting input to and/or output from a program. What you don't see all that often or that you may not be familiar with is redirecting I/O inside a bash script. And I'm not talking about redirections that you use when your script executes another command, I'm talking about redirecting your script's I/O once it has already started executing.
As an example, let's say that you want to add a --log option to your script and if the user specifies it you want all the output to go to a log file rather than to the stdout. Of course, the user could simply redirect the output on the command line, but let's assume there's a reason why that option doesn't appeal to you. So, to provide this feature in your script you can do:
#!/bin/bash echo hello # Parse command line options. # Execute the following if --log is seen. if test -t 1; then # Stdout is a terminal. exec >log else # Stdout is not a terminal, no logging. false fi echo goodbye echo error >&2
The if statement uses test to see if file descriptor number one is connected to a terminal (1 being the stdout). If it is then the exec command re-opens it for writing on the file named log. The exec command without a command but with redirections executes in the context of the current shell, it's the means by which you can open and close files and duplicate file descriptors. If file descriptor number one is not on a terminal then we don't change anything.
If you run this command you'll see the first echo and the last echo are output to the terminal. The first one happens before the redirection and the second one is specifically redirected to stderr (2 being stderr). So, how do you get stderr into the log file also? Just one simple change is required to the exec statement:
#!/bin/bash echo hello if test -t 1; then # Stdout is a terminal. exec >log 2>&1 else # Stdout is not a terminal, no logging. false fi echo goodbye echo error >&2
Here the exec statement re-opens stdout on the log file and then re-opens stderr on the same thing that stdout is opened on (this is how you duplicate file descriptors, aka dup them). Note that order is important here: if you change the order and re-open stderr first (i.e. exec 2>&1 >log), then it will still be on the terminal since you're opening it on the same thing stdout is on and at this point it's still the terminal.
Perhaps mainly as an exercise, let's try to do the same thing even if the output is not to the terminal. We can't do what we did above since re-opening stdout on the log file, when it's currently connected to a file redirection or to a pipeline, would break the redirection/pipeline that the user specified when the command was invoked.
Given the following command as an example:
bash test.sh | grep good
What we want to do is manipulate things so that it appears that the following command was executed instead:
bash test.sh | tee log | grep good
Your first thought might be that you could change the exec statement to something like this:
exec | tee log & # Won't work
and tell exec to re-open stdout on a background pipeline into tee, but that won't work (although bash doesn't complain about it). This just pipes exec's output to tee, and since exec doesn't produce any output in this instance, tee simply creates an empty file and exits.
You might also think that you can try some dup-ing of file descriptors and start tee in the background with it taking input from and writing output to different file descriptors. And you can do that, but the problem is that there's no way to create a new process that has its standard input connected to a pipe so that we can insert it into the pipeline (although see the postscript at the end of this article). If we could do this, the standard output of the tee command would be easy since by default that goes to the same place the main script's output goes to, so we could just close the main script's output and connected it to our pipe (if we just had a way to create it).
So are we at a dead end? Ahhhh no, that would be a different operating system you're thinking of. The solution is actually described in the last sentence of the previous paragraph. We just need a way to create a pipe, right? Well let's use named pipes.
#!/bin/bash echo hello if test -t 1; then # Stdout is a terminal. exec >log else # Stdout is not a terminal. npipe=/tmp/$$.tmp trap "rm -f $npipe" EXIT mknod $npipe p tee <$npipe log & exec 1>&- exec 1>$npipe fi echo goodbye
Here, if the script's stdout is not connected to the terminal, we create a named pipe (a pipe that exists in the file-system) using mknod and setup a trap to delete it on exit. Then we start tee in the background reading from the named pipe and writing to the log file. Remember that tee is also writing anything that it reads on its stdin to its stdout. Also remember that tee's stdout is also the same as the script's stdout (our main script, the one that invokes tee) so the output from tee's stdout is going to go wherever our stdout is currently going (i.e. to the user's redirection or pipeline that was specified on the command line). So at this point we have tee's output going where it needs to go: into the redirection/pipeline specified by the user.
Now all we need is to get tee reading the right data. And since tee is reading from a named pipe all we need to do is redirect our stdout to the named pipe. We close our current stdout (with exec 1>&-) and re-open it on the named pipe (with ezec 1>$npipe). Note that since tee is also writing to the redirection/pipeline that was specified on the command line, our closing the connection does not break anything.
Now if you run the command and pipe it's output to grep as above you'll see the output in the termimal and it will also be saved in the log file.
Many such journeys are possible, let the man page be your guide!
p.s. There's another way to do this using Bash 4's coproc statement, but that'll wait for another time.
Mitch Frazier is an Associate Editor for Linux Journal.
Fast/Flexible Linux OS Recovery
On Demand Now
In this live one-hour webinar, learn how to enhance your existing backup strategies for complete disaster recovery preparedness using Storix System Backup Administrator (SBAdmin), a highly flexible full-system recovery solution for UNIX and Linux systems.
Join Linux Journal's Shawn Powers and David Huffman, President/CEO, Storix, Inc.
Free to Linux Journal readers.Register Now!
- The Qt Company's Qt Start-Up
- Devuan Beta Release
- May 2016 Issue of Linux Journal
- EnterpriseDB's EDB Postgres Advanced Server and EDB Postgres Enterprise Manager
- The US Government and Open-Source Software
- Open-Source Project Secretly Funded by CIA
- The Humble Hacker?
- The Death of RoboVM
- New Container Image Standard Promises More Portable Apps
- BitTorrent Inc.'s Sync
In modern computer systems, privacy and security are mandatory. However, connections from the outside over public networks automatically imply risks. One easily available solution to avoid eavesdroppers’ attempts is SSH. But, its wide adoption during the past 21 years has made it a target for attackers, so hardening your system properly is a must.
Additionally, in highly regulated markets, you must comply with specific operational requirements, proving that you conform to standards and even that you have included new mandatory authentication methods, such as two-factor authentication. In this ebook, I discuss SSH and how to configure and manage it to guarantee that your network is safe, your data is secure and that you comply with relevant regulations.Get the Guide