Home ›

Normalizing Path Names with Bash

Sep 03, 2009  By Mitch Frazier
 in

The bash function presented here normalizes path names. By normalize I mean it removes unneeded /./ and ../dir sequences. For example, ../d1/./d2/../f1 normalized would be ../d1/f1.

The first version of the function uses bash regular expressions. The /./ sequences are removed first during variable expansion with substitution by the line:

local   path=${1//\/.\//\/}

The dir/.. sequences are removed by the loop:

while [[ $path =~ ([^/][^/]*/\.\./) ]]
do
    path=${path/${BASH_REMATCH[0]}/}
done

Each time a dir/.. match is found, variable expansion with substitution is used to remove the matched part of the path.

Regular expressions were introduced in bash 3.0. Bash 3.2 changed regular expression handling slightly in that quotes around regular expressions became part of the regular expression. So, if you have a version of bash (with regular expression support) and the code doesn't work, put the regular expression in the while loop in quotes.

The entire function and some test code follows:

#!/bin/bash
#
# Usage: normalize_path PATH
#
# Remove /./ and dir/.. sequences from a pathname and write result to stdout.

function normalize_path()
{
    # Remove all /./ sequences.
    local   path=${1//\/.\//\/}
    
    # Remove dir/.. sequences.
    while [[ $path =~ ([^/][^/]*/\.\./) ]]
    do
        path=${path/${BASH_REMATCH[0]}/}
    done
    echo $path
}

if [[ $(basename $0 .sh) == 'normalize_path' ]]; then
    if [[ "$*" ]]; then
        for p in "$@"
        do
            printf "%-30s => %s\n" $p $(normalize_path $p)
        done
    else
        for p in /test/../test/file test/../test/file .././test/../test/file
        do
            printf "%-30s => %s\n" $p $(normalize_path $p)
        done
    fi
fi


#####################################################################

# vim: tabstop=4: shiftwidth=4: noexpandtab:
# kate: tab-width 4; indent-width 4; replace-tabs false;

Since, older versions of bash don't support regular expressions the second version does the same thing using sed instead:

#!/bin/bash
#
# Usage: normalize_path PATH
#
# Remove /./ and dir/.. sequences from a pathname and write result to stdout.

function normalize_path()
{
    # Remove all /./ sequences.
    local   path=${1//\/.\//\/}
    
    # Remove first dir/.. sequence.
    local   npath=$(echo $path | sed -e 's;[^/][^/]*/\.\./;;')
    
    # Remove remaining dir/.. sequence.
    while [[ $npath != $path ]]
    do
        path=$npath
        npath=$(echo $path | sed -e 's;[^/][^/]*/\.\./;;')
    done
    echo $path
}

if [[ $(basename $(basename $0 .sh) .old) == 'normalize_path' ]]; then
    if [[ "$*" ]]; then
        for p in "$@"
        do
            printf "%-30s => %s\n" $p $(normalize_path $p)
        done
    else
        for p in /test/../test/file test/../test/file .././test/../test/file
        do
            printf "%-30s => %s\n" $p $(normalize_path $p)
        done
    fi
fi


#####################################################################

# vim: tabstop=4: shiftwidth=4: noexpandtab:
# kate: tab-width 4; indent-width 4; replace-tabs false;

You can run the script directly and it runs a few tests:

$ bash normalize_path.sh
/test/../test/file             => /test/file
test/../test/file              => test/file
.././test/../test/file         => ../test/file

You can also pass in test cases on the command line:

$ bash normalize_path.sh ../d1/./d2/../f1 a/b/c/../d/../e
../d1/./d2/../f1               => ../d1/f1
a/b/c/../d/../e                => a/b/e

Normalized path names are never necessary but they're often easier to comprehend at a glance.

______________________

Mitch Frazier is an Associate Editor for Linux Journal.

Comments

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

Using POSIX index arrays

Jon Disnard's picture
Submitted by Jon Disnard (not verified) on Mon, 10/04/2010 - 13:10.

I wrote about the same thing on my website.
https://sites.google.com/site/jdisnard/realpath

This is similar to what "S. Chauveau" except this is pure posix code.
Also, since the shell has positional parameters it does not require an array. Also this code will naively dereference a symbolic link.

No bash'isms, will work in KSH or any posix compliant shell.

real_path () {
  OIFS=$IFS
  IFS='/'
  for I in $1
  do
    # Resolve relative path punctuation.
    if [ "$I" = "." ] || [ -z "$I" ]
      then continue
    elif [ "$I" = ".." ]
      then FOO="${FOO%%/${FOO##*/}}"
           continue
      else FOO="${FOO}/${I}"
    fi

    # Dereference symbolic links.
    if [ -h "$FOO" ] && [ -x "/bin/ls" ]
      then IFS=$OIFS
           set `/bin/ls -l "$FOO"`
           while shift ;
           do
             if [ "$1" = "->" ]
               then FOO=$2
                    shift $#
                    break
             fi
           done
    fi
  done
  IFS=$OIFS
  echo "$FOO"
}

useful

Kendo's picture
Submitted by Kendo (not verified) on Thu, 09/23/2010 - 21:50.

You could also use parameter expansions for a portable solution, but I'm not sure how efficient that would be compared to a call to sed.

Absolute path

bhoejte's picture
Submitted by bhoejte (not verified) on Thu, 09/10/2009 - 06:48.

I have never had cause to normalize a path, but I have had cause to need an absolute path. It incidentally also seem to normalize the path as well.

Made in ksh, but seems to work in bash also. Not entirely shell script, as sed is a heavy component.

function AbsolutePath
{
  ABSFILE="${1:-$0}"
  {
  if [[ "${ABSFILE#/}" = "${ABSFILE}" ]]; then
    echo ${PWD}/${ABSFILE}
  else
    echo ${ABSFILE}
  fi
  } | sed '
    :a
    s;/\./;/;g
    s;//;/;g
    s;/[^/][^/]*/\.\./;/;g
    ta'
  unset ABSFILE
}

I'm quite certain that a better solution now exists in Linux (I mean... there must be...), but this was made 15/11-2002 on Tru64, and has worked since. Just be careful not to use an ABSFILE variable, as it will be unset at the end (not enough experience to localize it).

Too much code

Shantanu gadgil's picture
Submitted by Shantanu gadgil (not verified) on Fri, 09/04/2009 - 02:16.

I agree with previous poster ... I have never required this ... I rely on "readlink -f"

Arrays can also be used

S. Chauveau's picture
Submitted by S. Chauveau (not verified) on Fri, 09/04/2009 - 01:36.

Another way to something like that in bash is by using arrays and substrings.
I don't put the complete code but here is the idea:

# use this to split the path into parts using / as separator
OLD_IFS="$IFS" ; IFS="/" ; PARTS=($DIR) ; IFS="$OLD_IFS"

# Detect an absolute path by checking the 1st character
if [ "${DIR:0:1}" == "/" ] ; then PREFIX="/" else PREFIX="" fi

# Process the array according to your needs
nb=${#PARTS[*]}
i=0
for ((i=0;i<nb;i++)) ; do
part="${PARTS[$i]}"
if [ ... ] ; then
unset PARTS[$i]
fi
done

# and finally reconstruct the path

IFS='/' ; DIR="$PREFIX${PARTS[*]}" ; IFS="$OLD_IFS"

The advantage of that code over the 'sed' version is that it does not require to spawn any unix process.
It is pure bash.

Simpler way

Anatoli Sakhnik's picture
Submitted by Anatoli Sakhnik (not verified) on Fri, 09/04/2009 - 01:17.

For existing paths it'd be simpler to use `cd "$path" && pwd` or `readlink -f "$path"`.

Thank you for this suggestion

Anonymous Coward's picture
Submitted by Anonymous Coward (not verified) on Mon, 10/04/2010 - 18:43.

(cd $path && pwd ) is the suggestion I needed. Thanks!

I use that, but

TesserId's picture
Submitted by TesserId on Fri, 09/04/2009 - 09:54.

I've used that a lot. Sadly, it won't normalize relative paths. :(

Nice but... I cannot

Anonymous's picture
Submitted by Anonymous (not verified) on Thu, 09/03/2009 - 18:21.

Nice but... I cannot remember when in the past 8 years of me using unix-based operating systems, the need for such a script arose. No offense, though. Probably a good code example for shell scripting and regular expressions, but kind of "solution searches for problem".

+1

Ratso's picture
Submitted by Ratso (not verified) on Thu, 09/03/2009 - 22:57.

My thoughts exactly. I've read the article, and then in the end asked myself - when did I ever need this is my 10+ years of linux experience? Never! How can you even end up with 'bad' paths, if not by bad scripting? To me, this article is just another 'regex show off', which is sad - I almost never ever read other people regular expressions, there's no point - I either write them off from my head, or consult man pages some more and then write them ;) To me, there's no point in reading regex - you won't understand it if you don't now regex rules, but if you know the rules, you can write regex yourself. Only when bug-hunting I read "foreign" regex.

White Paper
More Resources
Fabric-Based Computing Enables Optimized Hyperscale Data Centers

Today’s modular x86 servers are compute-centric, designed as a least common denominator to support a wide range of IT workloads. Those generic, virtualized IT workloads have much different resource optimization requirements than hyperscale and cloud applications. They have resulted in a “one size fits all” enterprise IT architecture that is not optimized for a specific set of IT workloads, and especially not emerging hyperscale workloads, such as web applications, big data, and object storage. In this report, you will learn how shifting the focus from traditional compute-centric IT architectures to an innovative disaggregated fabric-based architecture can optimize and scale your data center.

Learn More

Sponsored by AMD

White Paper
More Resources
Red Hat White Paper: Using an Open Source Framework to Catch the Bad Guy

Built-in forensics, incident response, and security with Red Hat Enterprise Linux 6

Every security policy provides guidance and requirements for ensuring adequate protection of information and data, as well as high-level technical and administrative security requirements for a system in a given environment. Traditionally, providing security for a system focuses on the confidentiality of the information on it. However, protecting the data integrity and system and data availability is just as important. For example, when processing United States intelligence information, there are three attributes that require protection: confidentiality, integrity, and availability.

Learn more about catching the bad guy in this free white paper.

Learn More

Sponsored by DLT Solutions