Bash: Preserving Whitespace Using set and eval

November 5th, 2008 by Mitch Frazier in

Your rating: None Average: 3.7 (16 votes)

If you don't care much about whitespace bash is great: it normally turns multiple whitespace characters into one and it breaks things into words based on white space. If on the other hand you'd like to preserve whitespace bash can be a bit difficult at times. A trick which often helps is using a combination of bash's eval and set commands.

Let's say that you're building a list of items where each item may contain significant spaces, say something like:

#!/bin/bash

items=
for i in "$@"
do
    items="$items \"$i\""
done

for i in $items
do
    echo $i
done

But when you run this and try to use the items from the saved list you don't quite get what you expected:

  $ sh t1.sh "ab cd" "ef gh"
  "ab
  cd"
  "ef
  gh"

One solution is to do the following:

#!/bin/bash

items=
for i in "$@"
do
    items="$items \"$i\""
done

eval set -- $items
for i in "$@"
do
    echo $i
done

Which produces the desired result:

  $ sh t2.sh "ab cd" "ef gh"
  ab cd
  ef gh

The important line is:

  eval set -- $items

The set command takes any arguments after the options (here "--" signals the end of the options) and assigns them to the positional parameters ($0..$n). The eval command executes its arguments as a bash command.

If you do this without the eval command you'll get the same result as the first example. By passing the set command to eval bash will honor the embedded quotes in the string rather than assume they are part of the word.

If you run this script you can see a bit more of what bash is doing:

#!/bin/bash

items=
for i in "$@"
do
    items="$items \"$i\""
done

set -x
set -- $items
set +x
echo '===='
set -x
eval set -- $items
set +x

This produces:

  $ sh t3.sh "ab cd" "ef gh"
  + set -- '"ab' 'cd"' '"ef' 'gh"'
  + set +x
  ====
  + eval set -- '"ab' 'cd"' '"ef' 'gh"'
  ++ set -- 'ab cd' 'ef gh'
  + set +x
__________________________

Mitch Frazier is an Associate Editor for Linux Journal and the Web Editor for linuxjournal.com.


Special Magazine Offer -- Free Gift with Subscription
Receive a free digital copy of Linux Journal's System Administration Special Edition as well as instant online access to current and past issues. CLICK HERE for offer

Linux Journal: delivering readers the advice and inspiration they need to get the most out of their Linux systems since 1994.

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.
Gunstick's picture

DANGER!!! the eval executes the arguments

On September 15th, 2009 Gunstick (not verified) says:

try this:

t1.sh "ab cd" "ef \$(pwd) gh"

or

t1.sh "ab cd" 'ef $(pwd) gh'

Anonymous's picture

What about non-quoted?

On November 16th, 2008 Anonymous (not verified) says:

The IFS seems to fix the multiple space problem:

------------------------
#!/bin/bash

IFS=
while read line
do
echo $line
done
------------------------
This also does not need quoted input.

Anonymous's picture

On the example of reading from a file

On November 16th, 2008 Anonymous (not verified) says:

Suppose the user types
"ab"
"a b"
"a b"
"a b"
for stdin. We get:
ab
a b
a b
a b
I.e., multiple spaces are reduced to one space.

Anonymous's picture

Embedded Quotes

On November 6th, 2008 Anonymous (not verified) says:

The example given has problems with inputs that contain quotes

    $ t1.sh "foo says, \"bar\""

gives

    foo says, bar

To fix this, the inherent quote in each element needs to be replaced. The resulting script:

  #!/bin/bash

    items=
    for i in "$@"
    do
        items="$items \"$(echo $i | sed 's/"/\\"/g')\""
    done

    eval set -- $items
    for i in "$@"
    do
        echo $i
    done

gives the desired result:

foo says, "bar"

Anonymous's picture

Simple way

On November 6th, 2008 Anonymous (not verified) says:

A simple way would be to use an array :

items=("$@")
for i in "${items[@]}"; do
        echo $i
done

xavier

Mitch Frazier's picture

For this example

On November 6th, 2008 Mitch Frazier says:

Using an array is certainly an option in the example I used, but suppose the data with significant spaces is being read from a file or from the terminal such as in this example:

while read line
do
    eval set -- $line
    for i in "$@"
    do
        echo $i
    done
done

If the user types "ab  cd" "ef  gh" and we'd like to display ab  cd and ef  gh (without quotes) then you need eval. Another alternative that does use arrays is the following:

while read line
do
    eval items=($line)
    for i in "${items[@]}"
    do
        echo $i
    done
done
__________________________

Mitch Frazier is an Associate Editor for Linux Journal and the Web Editor for linuxjournal.com.

lening geld's picture

Thanks for clarifying that

On November 10th, 2008 lening geld (not verified) says:

Thanks for clarifying that part!

Post new comment

Please note that comments may not appear immediately, so there is no need to repost your comment.
The content of this field is kept private and will not be shown publicly.
  • Allowed HTML tags: <a> <em> <strong> <cite> <code> <pre> <ul> <ol> <li> <dl> <dt> <dd> <i> <b>
  • Lines and paragraphs break automatically.

More information about formatting options

Newsletter

Each week Linux Journal editors will tell you what's hot in the world of Linux. You will receive late breaking news, technical tips and tricks, and links to in-depth stories featured on www.linuxjournal.com.
Sign up for our Email Newsletter

Tech Tip Videos

From the Magazine

December 2009, #188

If last month's Infrastrucuture issue was too "big" for you then try on this month's Embedded issue. Find out how to use Player for programming mobile robots, build a humidity controller for your root cellar, find out how to reduce the boot time of your embedded system, and if you're new to embedded systems find out the basics that go into one. You can also read about the Beagle Board, the Mesh Potato and a spate of other interestingly named items. And along with our regular columns don't miss our new monthly column: Economy Size Geek.







Read this issue