Bash Scripting Tip: processing filenames with spaces
I can’t take credit for this. Stuart Jansen of Guru Labs referenced Tensai’s brilliance (nice play on words Stuart. For thoes who don’t know, ‘tensai’ in Japanese means smart or brilliant). Tensai is one of the members of PLUG and is also on irc.freenode.net’s #utah most of the time. Tensai in turn blames Hans Fugal for actually pointing him in the direction of this solution. And so we find that knowledge always exists, not unlike matter, neither destroyed or created, just changing forms. I wanted to get this down in my blog because it’s a good place to have ideas which are useful and hopefully it will help someone else.
Often one needs to process files which may contain spaces in them in a bash script. There are various ways to do this however, this is the simplest and most clean way that I have found to do this. This method uses the ‘read’ command which reads and entire line of text into a single variable. This varialble can then be quoted which protects the spaces as you process the files. Very simple and clean:
find | while read I; do echo "$I"; done
So, to copy some mp3s to a backup directory:
find /path/to/files/ -iname "*mp3" | \
while read I; do
cp -v --parent "$I" /backup/dir/
doneThis will copy all mp3 files under /path/to/files/ with their parent directory structure intact to /backup/dir/
Happy scripting.
Keywords: Linux, bash scripting, linux scripting, unix scripting, bash shell, shell scripting, unix shell programming, OSX.
You have a minor typo in your mp3 copying example: the first “$I” should just be “I”, as in your first example. So it should read:
find /path/to/files/ -iname “*mp3″ | \
while read I; do
cp -v –parent “$I” /backup/dir/
done
Comment by Darrell — 10/16/2005 @ 6:01 am
Thanks, fixed…
Comment by marc — 10/16/2005 @ 8:22 am
Thanks for this tip! I’d been agonizing over this for about 30 minutes until I found your site. Had I not it would have been trial and error all night until I got it right :)
Comment by Shawn Dowler — 1/24/2006 @ 9:25 pm
Hey Shawn,
I appreciate the comment! I’m glad it helped - that’s exactly why I wrote it up :)
Comment by marc — 1/24/2006 @ 9:39 pm
Great!! Perfect, to pipe find into the loop. Tricky. Thanks.
Comment by fabe — 2/25/2006 @ 3:44 am
nice trick. find is nice because it has soo many options, but if all we need is a set of files matching a pattern,
find $pathtompgs -iname ‘*.mpg’ | while read I; do echo “$I”; done
could be replaced with:
for I in $pathtompgs/*.mpg; do echo “$I”; done
(bash can do some of the work)
Comment by Pablo — 3/29/2007 @ 11:57 pm
Hi Pablo, that actually doesn’t work with file names or paths with spaces which is the reason for the post in the first place. Reading each line into a variable with the described method protects the spaces in path and file names. It would work if your paths/filenames don’t have spaces though.
Thanks.
Comment by marc — 3/30/2007 @ 3:46 am
All is fair in love and war, but why wont my script work? :(
It only iterates over the first occurance.
~/music/ $ find ./ -iname “*.flac” | while read I; do ffmpeg -ab 160k -i “$I” “$I.mp3″; mv “$I.mp3″ mp3/; done;
Comment by Johan — 8/20/2008 @ 11:48 pm