Bash: Preserving Whitespace Using set and eval

 in

If you don't care much about whitespace bash is great: it normally turns multiple whitespace characters into one and it breaks things into words based on white space. If on the other hand you'd like to preserve whitespace bash can be a bit difficult at times. A trick which often helps is using a combination of bash's eval and set commands.

Let's say that you're building a list of items where each item may contain significant spaces, say something like:

#!/bin/bash

items=
for i in "$@"
do
    items="$items \"$i\""
done

for i in $items
do
    echo $i
done

But when you run this and try to use the items from the saved list you don't quite get what you expected:

  $ sh t1.sh "ab cd" "ef gh"
  "ab
  cd"
  "ef
  gh"

One solution is to do the following:

#!/bin/bash

items=
for i in "$@"
do
    items="$items \"$i\""
done

eval set -- $items
for i in "$@"
do
    echo $i
done

Which produces the desired result:

  $ sh t2.sh "ab cd" "ef gh"
  ab cd
  ef gh

The important line is:

  eval set -- $items

The set command takes any arguments after the options (here "--" signals the end of the options) and assigns them to the positional parameters ($0..$n). The eval command executes its arguments as a bash command.

If you do this without the eval command you'll get the same result as the first example. By passing the set command to eval bash will honor the embedded quotes in the string rather than assume they are part of the word.

If you run this script you can see a bit more of what bash is doing:

#!/bin/bash

items=
for i in "$@"
do
    items="$items \"$i\""
done

set -x
set -- $items
set +x
echo '===='
set -x
eval set -- $items
set +x

This produces:

  $ sh t3.sh "ab cd" "ef gh"
  + set -- '"ab' 'cd"' '"ef' 'gh"'
  + set +x
  ====
  + eval set -- '"ab' 'cd"' '"ef' 'gh"'
  ++ set -- 'ab cd' 'ef gh'
  + set +x
______________________

Mitch Frazier is an Associate Editor for Linux Journal.

Comments

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

I dont think I am quite getting eval with arrays!

Anonymous's picture

How about pulling lines from a file into an array while retaining whitespaces on each line? For example:

___________________________________________________________________
#releases.txt
#
CentOS release 5.2 (Final)
CentOS release 5.3 (Final)
CentOS release 5.4 (Final)
Fedora Core release 4 (Stentz)
Fedora Core release 5 (Bordeaux)
Fedora Core release 6 (Zod)
Fedora release 7 (Moonshine)
Fedora release 8 (Werewolf)
Fedora release 9 (Sulphur)
Red Hat Enterprise Linux AS release 4 (Nahant Update 2)
Red Hat Enterprise Linux ES release 3 (Taroon Update 1)
Red Hat Linux release 8.0 (Psyche)
Red Hat Linux release 9 (Shrike)
Ubuntu 2.6.22-14.46-server
Ubuntu 2.6.31-15.50-generic
____________________________________________________________________

so if your array was MYARRAY and the lines where as shown above then $MYARRAY[0] would equal: CentOS release 5.2 (Final)

Would eval work for this or am I barking up the wrong tree?

I guess technically in that

Anonymous's picture

I guess technically in that little file $MYARRAY[0] would equal "#releases.txt" and $MYARRAY[2] would equal: CentOS release 5.2 (Final)

But i think you can get what I am trying to say!

DANGER!!! the eval executes the arguments

Gunstick's picture

try this:

t1.sh "ab cd" "ef \$(pwd) gh"

or

t1.sh "ab cd" 'ef $(pwd) gh'

What about non-quoted?

Anonymous's picture

The IFS seems to fix the multiple space problem:

------------------------
#!/bin/bash

IFS=
while read line
do
echo $line
done
------------------------
This also does not need quoted input.

On the example of reading from a file

Anonymous's picture

Suppose the user types
"ab"
"a b"
"a b"
"a b"
for stdin. We get:
ab
a b
a b
a b
I.e., multiple spaces are reduced to one space.

Embedded Quotes

Anonymous's picture

The example given has problems with inputs that contain quotes

    $ t1.sh "foo says, \"bar\""

gives

    foo says, bar

To fix this, the inherent quote in each element needs to be replaced. The resulting script:

  #!/bin/bash

    items=
    for i in "$@"
    do
        items="$items \"$(echo $i | sed 's/"/\\"/g')\""
    done

    eval set -- $items
    for i in "$@"
    do
        echo $i
    done

gives the desired result:

foo says, "bar"

Simple way

Anonymous's picture

A simple way would be to use an array :

items=("$@")
for i in "${items[@]}"; do
        echo $i
done

xavier

For this example

Mitch Frazier's picture

Using an array is certainly an option in the example I used, but suppose the data with significant spaces is being read from a file or from the terminal such as in this example:

while read line
do
    eval set -- $line
    for i in "$@"
    do
        echo $i
    done
done

If the user types "ab  cd" "ef  gh" and we'd like to display ab  cd and ef  gh (without quotes) then you need eval. Another alternative that does use arrays is the following:

while read line
do
    eval items=($line)
    for i in "${items[@]}"
    do
        echo $i
    done
done

Mitch Frazier is an Associate Editor for Linux Journal.

Thanks for clarifying that

lening geld's picture

Thanks for clarifying that part!

White Paper
Linux Management with Red Hat Satellite: Measuring Business Impact and ROI

Linux has become a key foundation for supporting today's rapidly growing IT environments. Linux is being used to deploy business applications and databases, trading on its reputation as a low-cost operating environment. For many IT organizations, Linux is a mainstay for deploying Web servers and has evolved from handling basic file, print, and utility workloads to running mission-critical applications and databases, physically, virtually, and in the cloud. As Linux grows in importance in terms of value to the business, managing Linux environments to high standards of service quality — availability, security, and performance — becomes an essential requirement for business success.

Learn More

Sponsored by Red Hat

White Paper
Private PaaS for the Agile Enterprise

If you already use virtualized infrastructure, you are well on your way to leveraging the power of the cloud. Virtualization offers the promise of limitless resources, but how do you manage that scalability when your DevOps team doesn’t scale? In today’s hypercompetitive markets, fast results can make a difference between leading the pack vs. obsolescence. Organizations need more benefits from cloud computing than just raw resources. They need agility, flexibility, convenience, ROI, and control.

Stackato private Platform-as-a-Service technology from ActiveState extends your private cloud infrastructure by creating a private PaaS to provide on-demand availability, flexibility, control, and ultimately, faster time-to-market for your enterprise.

Learn More

Sponsored by ActiveState