Bash Redirections Using Exec
If you've used the command line much at all you know about I/O redirection for redirecting input to and/or output from a program. What you don't see all that often or that you may not be familiar with is redirecting I/O inside a bash script. And I'm not talking about redirections that you use when your script executes another command, I'm talking about redirecting your script's I/O once it has already started executing.
As an example, let's say that you want to add a --log option to your script and if the user specifies it you want all the output to go to a log file rather than to the stdout. Of course, the user could simply redirect the output on the command line, but let's assume there's a reason why that option doesn't appeal to you. So, to provide this feature in your script you can do:
#!/bin/bash echo hello # Parse command line options. # Execute the following if --log is seen. if test -t 1; then # Stdout is a terminal. exec >log else # Stdout is not a terminal, no logging. false fi echo goodbye echo error >&2
The if statement uses test to see if file descriptor number one is connected to a terminal (1 being the stdout). If it is then the exec command re-opens it for writing on the file named log. The exec command without a command but with redirections executes in the context of the current shell, it's the means by which you can open and close files and duplicate file descriptors. If file descriptor number one is not on a terminal then we don't change anything.
If you run this command you'll see the first echo and the last echo are output to the terminal. The first one happens before the redirection and the second one is specifically redirected to stderr (2 being stderr). So, how do you get stderr into the log file also? Just one simple change is required to the exec statement:
#!/bin/bash echo hello if test -t 1; then # Stdout is a terminal. exec >log 2>&1 else # Stdout is not a terminal, no logging. false fi echo goodbye echo error >&2
Here the exec statement re-opens stdout on the log file and then re-opens stderr on the same thing that stdout is opened on (this is how you duplicate file descriptors, aka dup them). Note that order is important here: if you change the order and re-open stderr first (i.e. exec 2>&1 >log), then it will still be on the terminal since you're opening it on the same thing stdout is on and at this point it's still the terminal.
Perhaps mainly as an exercise, let's try to do the same thing even if the output is not to the terminal. We can't do what we did above since re-opening stdout on the log file, when it's currently connected to a file redirection or to a pipeline, would break the redirection/pipeline that the user specified when the command was invoked.
Given the following command as an example:
bash test.sh | grep good
What we want to do is manipulate things so that it appears that the following command was executed instead:
bash test.sh | tee log | grep good
Your first thought might be that you could change the exec statement to something like this:
exec | tee log & # Won't work
and tell exec to re-open stdout on a background pipeline into tee, but that won't work (although bash doesn't complain about it). This just pipes exec's output to tee, and since exec doesn't produce any output in this instance, tee simply creates an empty file and exits.
You might also think that you can try some dup-ing of file descriptors and start tee in the background with it taking input from and writing output to different file descriptors. And you can do that, but the problem is that there's no way to create a new process that has its standard input connected to a pipe so that we can insert it into the pipeline (although see the postscript at the end of this article). If we could do this, the standard output of the tee command would be easy since by default that goes to the same place the main script's output goes to, so we could just close the main script's output and connected it to our pipe (if we just had a way to create it).
So are we at a dead end? Ahhhh no, that would be a different operating system you're thinking of. The solution is actually described in the last sentence of the previous paragraph. We just need a way to create a pipe, right? Well let's use named pipes.
#!/bin/bash echo hello if test -t 1; then # Stdout is a terminal. exec >log else # Stdout is not a terminal. npipe=/tmp/$$.tmp trap "rm -f $npipe" EXIT mknod $npipe p tee <$npipe log & exec 1>&- exec 1>$npipe fi echo goodbye
Here, if the script's stdout is not connected to the terminal, we create a named pipe (a pipe that exists in the file-system) using mknod and setup a trap to delete it on exit. Then we start tee in the background reading from the named pipe and writing to the log file. Remember that tee is also writing anything that it reads on its stdin to its stdout. Also remember that tee's stdout is also the same as the script's stdout (our main script, the one that invokes tee) so the output from tee's stdout is going to go wherever our stdout is currently going (i.e. to the user's redirection or pipeline that was specified on the command line). So at this point we have tee's output going where it needs to go: into the redirection/pipeline specified by the user.
Now all we need is to get tee reading the right data. And since tee is reading from a named pipe all we need to do is redirect our stdout to the named pipe. We close our current stdout (with exec 1>&-) and re-open it on the named pipe (with ezec 1>$npipe). Note that since tee is also writing to the redirection/pipeline that was specified on the command line, our closing the connection does not break anything.
Now if you run the command and pipe it's output to grep as above you'll see the output in the termimal and it will also be saved in the log file.
Many such journeys are possible, let the man page be your guide!
p.s. There's another way to do this using Bash 4's coproc statement, but that'll wait for another time.
Mitch Frazier is an Associate Editor for Linux Journal.
|Happy Birthday Linux||Aug 25, 2016|
|ContainerCon Vendors Offer Flexible Solutions for Managing All Your New Micro-VMs||Aug 24, 2016|
|Updates from LinuxCon and ContainerCon, Toronto, August 2016||Aug 23, 2016|
|NVMe over Fabrics Support Coming to the Linux 4.8 Kernel||Aug 22, 2016|
|What I Wish I’d Known When I Was an Embedded Linux Newbie||Aug 18, 2016|
|Pandas||Aug 17, 2016|
- Scalable OpenGroupware.org
- Linux for the International Space Station Program
- eCrash: Debugging without Core Dumps
- Suggestions For The Greatest Automobile Insurance Offer
- Windows/Linux Dual Boot
- Linux and the Next Generation Internet
- Downloading an Entire Web Site with wget
- What Will My New Laptop Be?
- Returning Values from Bash Functions
- Paranoid Penguin - DNS Cache Poisoning, Part I
With all the industry talk about the benefits of Linux on Power and all the performance advantages offered by its open architecture, you may be considering a move in that direction. If you are thinking about analytics, big data and cloud computing, you would be right to evaluate Power. The idea of using commodity x86 hardware and replacing it every three years is an outdated cost model. It doesn’t consider the total cost of ownership, and it doesn’t consider the advantage of real processing power, high-availability and multithreading like a demon.
This ebook takes a look at some of the practical applications of the Linux on Power platform and ways you might bring all the performance power of this open architecture to bear for your organization. There are no smoke and mirrors here—just hard, cold, empirical evidence provided by independent sources. I also consider some innovative ways Linux on Power will be used in the future.Get the Guide