Variable Mangling in Bash with String Operators

Editor's Note: This article has been updated by its author.
Thank you, Pat.

Have you ever wanted to change the names of many files at once? Or, have
you ever needed to use a default value for a variable that has no value?
These and many other options are available to you when you use string
operators in bash and other Bourne-derived shells.

String operators allow you to manipulate the contents of a variable
without having to write your own shell functions to do so. They are
provided through "curly brace" syntax. Any variable can be displayed as
${foo} without changing its meaning. This functionality often is used to
protect a variable name from surrounding characters.

$ export foo=foo 
$ echo ${foo}bar # foo exists so this works as expected
foobar
$ echo $foobar # foobar doesn't exist, so this doesn't

$

By the end of this article, you'll be able to use it for a whole lot more.

Three kinds of variable substitution are available for use: pattern
matching, substitution and command substitution. I talk about the first
two variables here and leave command substitution for another time.
Pattern Matching
In pattern matching, you can match from the left or from the right. The
operators, along with their functions and examples, are shown in the
following table:

Operator: ${foo#t*is}

Function: deletes the shortest possible match from the left

Example:

$ export foo="this is a test"
$ echo ${foo#t*is}
is a test
$

Operator: ${foo##t*is}

Function: deletes the longest possible match from the left

Example:

$ export foo="this is a test"
$ echo ${foo##t*is}
a test
$

Operator: ${foo%t*st}
Function: deletes the shortest
possible match from the right
Example:

$ export foo="this is a test"
$ echo ${foo%t*st}
this is a
$

Operator: ${foo%%t*st}

Function: deletes the longest
possible match from the right
Example:

$ export foo="this is a test"
$ echo ${foo%%t*st}

$

Although the # and % identifiers may not seem obvious, they have a
convenient mnemonic. The # key is on the left side of the $ key and
operates from the left. The % key is on the right of the $ key and
operates from the right. (This is true, at least, for US qwerty keyboards.)

The operators listed above can be used to do a variety of things. For
example, the following script changes the extension of all .html files
so they now are .htm files.

#!/bin/bash 
# quickly convert html filenames for use on a dossy system
# only handles file extensions, not file names
for i in *.html; do
   if [ -f ${i%l} ]; then 
      echo "${i%l} already exists"
   else 
      mv $i ${i%l}
   fi 
done

Substitution
Another kind of variable mangling you might want to employ is
substitution. Four substitution operators are used in Bash, and they
are shown below:

Operator: ${foo:-bar}

Function: If $foo exists
and is not null, return $foo. If it doesn't exist or is null,
return bar.

Example:

$ export foo=""
$ echo ${foo:-one}
one
$ echo $foo

$

Operator: ${foo:=bar}

Function: If $foo exists and is
not null, return $foo. If it doesn't exist or is null, set $foo
to bar and return bar.

Example:

$ export foo=""
$ echo ${foo:=one}
one

$ echo $foo
one
$

Operator: ${foo:+bar}

Function: If $foo exists and is not null, return bar. If it doesn't exist
or is null, return a null.

Example:

$ export foo="this is a test"
$ echo ${foo:+bar}
bar
$

Operator: ${foo:?"error message"}

Function: If $foo exists and isn't null, return its value. If it doesn't
exist or is null, print the error message. If no error message is given,
it prints parameter null or not
set. In a non-interactive shell, this
aborts the current script. In an interactive shell, this simply prints
the error message.

Example:

$ export foo="one"
$ for i in foo bar baz; do
> eval echo \${$i:?}
> done
one
bash: bar: parameter null or not set
bash: baz: parameter null or not set
$

The : in the above operators can be omitted. Doing so changes the
behavior of the operator so that it simply tests for the existence of
the variable. This, in turn, causes the creation of a variable, for
example:

$ export foo="this is a test"
$ echo $bar

$ echo ${foo=bar}
this is a test
$ echo ${bar=bar}
bar
$ echo $bar
bar
$          

These operators can be used in a variety of ways. A good example would
be, in the case when no arguments are given, to give a default value to
a variable normally read from command-line arguments. This example is
demonstrated in the following script:

#!/bin/bash 
export INFILE=${1-"infile"} 
export OUTFILE=${2-"outfile"}
cat $INFILE > $OUTFILE

Copyright (c) 2005, 2000 by Pat Eyler. Originally published in Linux
Gazette issue 67. Copyright (c) 2000, Specialized Systems Consultants,
Inc. The material in this article may be distributed only subject to the
terms and conditions set forth in the Open Publication License, v1.0 or
later.