Home ›

Work the Shell - Dealing with Spaces in Filenames

Issue 202

From Issue #202
February 2011

Mar 31, 2011  By Dave Taylor
 in
How do you deal with this problem in your scripts?

and reversed with: 

original="$(echo $safename | sed s'/_-_/ /g')"

It solves the problem, but it's definitely not a very efficient or smart use of computing resources.

I've outlined three possible solution paths herein: modifying the IFS value, ensuring that you always quote filenames where referenced, and rewriting filenames internally to replace spaces with unlikely character sequences, reversing it on your way out of the script.

By the way, have you ever tried using the find|xargs pair with filenames that have spaces? It's sufficiently complicated that modern versions of these two commands have special arguments to denote that spaces might appear as part of the filenames: find -print and xargs -0 (and typically, they're not the same flags, but that's another story).

During the years I've been writing this column, I've more than once tripped up on this particular problem and received e-mail messages from readers sharing how a sample script tosses up its bits when a file with a space in its name appears. They're right.

My defensive reaction is “dude, don't use spaces in filenames”, but that's not really a good long-term solution, is it?

What I'd like to do instead is open this up for discussion on the Linux Journal discussion boards: how do you solve this problem within your scripts? Or, do you just religiously avoid using spaces in your filenames?

Dave Taylor has been hacking shell scripts for a really long time, 30 years. He's the author of the popular Wicked Cool Shell Scripts and can be found on Twitter as @DaveTaylor and more generally at www.DaveTaylorOnline.com.

______________________

Dave Taylor has been hacking shell scripts for over thirty years. Really. He's the author of the popular "Wicked Cool Shell Scripts" and can be found on Twitter as @DaveTaylor and more generally at www.DaveTaylorOnline.com.

Comments

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

Why not using the IFS??

Saladin's picture
Submitted by Saladin (not verified) on Thu, 04/21/2011 - 03:51.

I've gone through this dilemma in recent project and found using IFS simple and very effective,

Is this possible for all cases and the one mentioned in the article above?

IFS_backup=$IFS

IFS=$(echo -en "\n\b")

#do what ever you want ... then restore the original variable

IFS=$IFSBU

--

Salah Aldeen

Regards,

The problem, internal shell conversions

xae's picture
Submitted by xae (not verified) on Tue, 04/05/2011 - 14:32.

To manage this problem I had been using code like this,


for filename in *; do aux=$filename/; LS[${#LS[@]}]=${aux%/}; done
...
rm "${LS[24]}", as example

The catch here is to avoid the conversions taken by the shell when you access to the content of a variable, the trick is to append to the variable a character, trying to make the shell guess that it's a string, and then you can gain access without infamous conversions, but at later you ougth to remove the char in order to use it as a filename.

The chars candidates for this job are NUll, $'/0, and "/".

s/find -print/find -print0/ ?

mattcen's picture
Submitted by mattcen on Sat, 03/26/2011 - 18:04.

Dave, did you mean '-print0' as opposed to '-print'? While both kind of make sense, when paired with 'xargs -0' I think the former makes *more* sense.

With respect to spaces, I usually don't bother dealing with them unless I *know* that they'll be there (e.g. I'm manipulating data that non-geek customers access via a Samba share and put plenty of weird characters in).

One of the other solutions I've considered, but not really had to use yet, is to switch to Perl or Python for these particular use-cases, given they handle escaping of spaces somewhat better.

It's something I'd rather not do given I'm more familiar with Bash than either of the above, but it may be the lease bad solution, even if I do have to dig up my text books to work out how to do something that would be trivial in Bash.

--
Regards,
Matthew Cengia

Webcast
More Resources
How to Build an Optimal Hadoop Cluster to Store and Maintain Unlimited Amounts of Data Using Microservers

Realizing the promise of Apache® Hadoop® requires the effective deployment of compute, memory, storage and networking to achieve optimal results. With its flexibility and multitude of options, it is easy to over or under provision the server infrastructure, resulting in poor performance and high TCO. Join us for an in depth, technical discussion with industry experts from leading Hadoop and server companies who will provide insights into the key considerations for designing and deploying an optimal Hadoop cluster.

Learn More

Sponsored by AMD

White Paper
More Resources
Red Hat White Paper: Using an Open Source Framework to Catch the Bad Guy

Built-in forensics, incident response, and security with Red Hat Enterprise Linux 6

Every security policy provides guidance and requirements for ensuring adequate protection of information and data, as well as high-level technical and administrative security requirements for a system in a given environment. Traditionally, providing security for a system focuses on the confidentiality of the information on it. However, protecting the data integrity and system and data availability is just as important. For example, when processing United States intelligence information, there are three attributes that require protection: confidentiality, integrity, and availability.

Learn more about catching the bad guy in this free white paper.

Learn More

Sponsored by DLT Solutions