Bash Parameter Expansion

 in

If you use bash you already know what Parameter Expansion is, although you may have used it without knowing its name. Anytime you use a dollar sign followed by a variable name you're doing what bash calls Parameter expansion, eg echo $a or a=$b. But parameter expansion has numerous other forms which allow you to expand a parameter and modify the value or substitute other values in the expansion process.

Parameter expansion comes in many forms in bash, the simplest is just a dollar sign followed by a name, eg $a. This form merely substitutes the value of the variable in place of the parameter expansion expression. The variable name can also optionally be surround by braces, eg ${a}. If the variable name is immediately followed by characters that could be part of a variable name then the braces are needed to delimit the variable name, for example if you remove the braces from echo ${a}bc bash will try to expand the variable "abc" rather than "a".

One useful form of parameter expansion is to use a default value for a variable if it is not set. This is done with the syntax: ${VAR:-DFLT}. You might use this to allow your code to be modified via variables from the environment. Consider the following from a script, call it test.sh:

  TEST_MODE=${TEST_MODE:-0}
  ...
  if [[ $TEST_MODE -eq 0 ]]; then
      echo "Running in live mode"
  else
      echo "Running in test mode"
  fi
Normally the script runs in "live" mode but if you run it via:
  $ env TEST_MODE=1 sh test.sh
it runs in test mode.

You might also use the default value expansion with command line arguments or values from a config file, for example:

  # set cmd_param_x to 1 if seen on the command line
  ...
  if [[ ${cmd_param_x:-0} -eq 0 ]]; then
      echo "-x not specified"
  else
      echo "-x specified"
  fi

Another useful form of parameter expansion is to expand a variable and do string substitution on the value using the form ${VAR/search/replace}. For example:

  VAR=aabbcc
  echo ${VAR/b/-dd-}
outputs "aa-dd-bcc". Note that only the first instance of the search string is replaced, if you want to replace all instances use a double slash:
  VAR=aabbcc
  echo ${VAR//b/-dd-}
which now outputs "aa-dd--dd-cc".

There are also expansions for removing prefixes and suffixes. The form ${VAR#pattern} removes any prefix from the expanded value that matches the pattern. The removed prefix is the shortest matching prefix, if you use double pound-signs/hash-marks the longest matching prefix is removed. Similarily, the form ${VAR%pattern} removes a matching suffix (single percent for the shortest suffix, double for the longest). For example:

  file=data.txt
  echo ${file%.*}
  echo ${file#*.}
outputs the file base and extension respectively ("data" and "txt").

Note: if you have trouble remembering which is which of these two syntaxes, the "#" is to the left of the "%" key on your keyboard, just as prefixes come before suffixes. Also note that these are glob patterns not regular expressions.

Another expansion that exists is to extract substrings from the expanded value using the form ${VAR:offset:length}. This works in the expected form: offsets start at zero, if you don't specify a length it goes to the end of the string. For example:

  str=abcdefgh
  echo ${str:0:1}
  echo ${str:1}
outputs "a" and "bcdefgh".

This form also accepts negative offsets which count backwards from the end of the string. So this:

  str=abcdefgh
  echo ${str:-3:2}
produces "abcdefgh"... oops, what happened there? What happened was that bash misinterpretted what we wanted because the expansion looks like a default value expansion: ${VAR:-DFLT}. First time I tried this I stared at it for quite a while before a light came on as to how to do it (without using a variable [see below]):
  str=abcdefgh
  echo ${str:$((-3)):2}
which outputs the desired value "fg". The "$((...))" causes bash to treat the value as an arithmetic expansion (ie a number). Another slightly longer way of doing this is:
  str=abcdefgh
  i=-3
  echo ${str:$i:2}

The final form of parameter expansion I want to mention is one which simply expands to the length of the variable's value, its form is ${#VAR}. So for example:

  str=abcdef
  echo ${#str}
outputs "6".

Using these forms of parameter expansion in your shell scripts can simplify and shorten your scripts. These are not the only forms of parameter expansion that bash supports but they're the ones that I've found most useful over time. For more information see the "Parameter Expansion" section of the bash man page.

p.s. Note that all of the above forms of parameter expansion also work with bash's Special parameters: "$$", "$0", "$1", etc.

______________________

Mitch Frazier is an Associate Editor for Linux Journal.

Comments

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

Controlling the order of expansion

Anonymous's picture

I've got a bunch of image tiles which I want to put back together as the original image. They're named like img_x{0..20}_y{0..5}.jpg.

ImageMagick provides a neat little tool called montage. I'd like to use it like so:

montage img_x{0..20}_y{0..5}.jpg -geometry 0+0 -tile 21x6 whole.jpg.

The problem is that the order of the expanded arguments is wrong. I want to list all the y0 ones, then the y1 ones, etc.

I know a bunch of different tricks I can use to list those names in the right order (for loops, perl, etc), but is there a way to do it with bash's brace expansion?

Multiple Operations?

profylno's picture

I've wanted to know for a long time if it is possible to use a combination of these operations, for example, to make two substitutions. I find it so annoying when I still have to use sed for a second substitution after a nice little bash expansion.

No

Mitch Frazier's picture

You mean something like:

  a='hello world'
  a=${${a/hello/goodbye}/world/earth}   # NO GOOD

That doesn't work, because bash expects a variable name after "${" not a value. Although I'm not sure why you would use sed instead of just using a second bash substitution, eg:

  a='hello world'
  a=${a/hello/goodbye}
  a=${a/world/earth}

But maybe I'm misunderstanding your question.

Mitch Frazier is an Associate Editor for Linux Journal.

You understand my question

profylno's picture

You understand my question perfectly.
As for your suggestion:
Suppose I have a script that takes an argument that will always be of the form CRAPimportantstuff, and I wanted to use it just once, I could say

  blablabla ${1#CRAP}

I wouldn't have to set somevar to ${1#CRAP} and then say 'blablabla $somevar', which is something I like very much about bash parameter expansion. But if the argument was in the form CRAPimportanstuffCRAP, this wouldn't fly. Now, I could, as you suggested, introduce another variable (or just change $1, though I'm not sure if that's possible), but I usually just end up doing

  blablabla `echo -n ${1#CRAP} | sed 's/CRAP$//'`

or something similarly ugly, because I don't feel like setting a variable to a value I'm only going to use once. Perhaps this is a bad habit on my part, and I should just do as you suggest. I just find it annoying that there's nothing that can handle the second case in a way as pretty and compact as the first (or, as was my original question, is there?).

You could try a regular expression

Anonymous's picture

If your expression is complicated enough or your aversion to specifying a temp variable is strong enough, you can try bash 3's regular expression facility:

[[ $1 =~ ^prefix(.*)suffix$ ]]
blablabla ${BASH_REMATCH[1]}

Though in profylno's particular case I would simply do the temp variable.

Don't think so

Mitch Frazier's picture

No easy way to do that comes to mind.

Mitch Frazier is an Associate Editor for Linux Journal.

Keyboard-based mnemonics...

peter.green's picture

...don't travel well, as I've pointed out before on LJ. My UK keyboard has the # way over to the right of the % (and two rows down!)

Off the top of my head, a less US-centric reminder might be the well-known game Tic-Tac-Toe, aka Noughts-and-Crosses. The # resembles the game grid and the % the noughts: in play, the grid comes first, then the noughts.

Edit: Having slept on it, an even simpler mnemonic (for English-speakers) is that alphabetically, H for Hash comes before P for Percent.

Nice tips...

Gnusci's picture

This tips will really help me a lot, much simpler than other ways, Thanks

Prefix / Suffix example

flatcap's picture

You might want to use the double # format if there could be more than one '.' in the filename.

FILE=some.data.txt

echo ${FILE%.*}       # "some.data"
echo ${FILE#*.}       # "data.txt"

echo ${FILE%.*}       # "some.data"
echo ${FILE##*.}      # "txt"

Negative indices....

Boscorama's picture

In the 'trailing' sub-string example, just place a space before the negative number:

str=abcdefgh
echo ${str: -3:2}

outputs "fg" as desired.

or echo ${str:(-3):2}

wodny's picture

or echo ${str:(-3):2}

Much simpler than my solution

Mitch Frazier's picture

Thanks

Mitch Frazier is an Associate Editor for Linux Journal.

Very useful tips! Now I

Fahd Shariff's picture

Very useful tips!
Now I won't have to echo my variables into sed or awk anymore!

White Paper
Linux Management with Red Hat Satellite: Measuring Business Impact and ROI

Linux has become a key foundation for supporting today's rapidly growing IT environments. Linux is being used to deploy business applications and databases, trading on its reputation as a low-cost operating environment. For many IT organizations, Linux is a mainstay for deploying Web servers and has evolved from handling basic file, print, and utility workloads to running mission-critical applications and databases, physically, virtually, and in the cloud. As Linux grows in importance in terms of value to the business, managing Linux environments to high standards of service quality — availability, security, and performance — becomes an essential requirement for business success.

Learn More

Sponsored by Red Hat

White Paper
Private PaaS for the Agile Enterprise

If you already use virtualized infrastructure, you are well on your way to leveraging the power of the cloud. Virtualization offers the promise of limitless resources, but how do you manage that scalability when your DevOps team doesn’t scale? In today’s hypercompetitive markets, fast results can make a difference between leading the pack vs. obsolescence. Organizations need more benefits from cloud computing than just raw resources. They need agility, flexibility, convenience, ROI, and control.

Stackato private Platform-as-a-Service technology from ActiveState extends your private cloud infrastructure by creating a private PaaS to provide on-demand availability, flexibility, control, and ultimately, faster time-to-market for your enterprise.

Learn More

Sponsored by ActiveState