Testing Filename Expansion

The Problem

There's a lot of code that does something like:

% cat *.txt

which seems innocuous enough but from time to time you'll see:

% cat *.txt
cat: cannot open *.txt: No such file or directory

because there are no files called *.txt.

Furthermore, cat has exited with the value 2 -- which you should care about if you are handling errors properly.

Whatever the reasons for there being no files we should be able to handle this situation a little bit more gracefully.

The Reason

When the shell expands a wildcard if there are no files that match the wildcard it simply returns the wildcard itself. It is then the command, cat, that tries to open a literal file called *.txt and fails. If the shell had simply returned nothing you would get the following:

% cat
[a very long pause until you realise cat is trying to read its stdin]

The Solution

We need to get in the way and see whether the result of filename expansion has returned the names of any real files. This becomes a three step process

1. capture the filename expansion
2. test for the presence of any files
3. run cat

Capturing the Filename Expansion

This seems a bit heavyweight but we can choose one of two ways, both of which manipulate an array:

% txt_filenames=( *.txt )

% set -- *.txt

Test For the Presence of Files

If we were testing for files we can use the -f operator or perhaps -d if we were testing for directories, etc..

Do we need to test every element in the array? No. Remember that if filename expansion was successful we'll have an array whose elements are file1.txt, file2.txt, file3.txt etc. all of which are files and if it was unsuccessful we'll have an array whose sole element is *.txt. Either way, we'll have something in the first element of the array which we can run the -f operator against which will tell us if the rest of the array is worth looking at:

% [[ -f "${txt_filenames[0]}" ]]

or

% [[ -f "$1" ]]

If that first element is a real file then [[ return success but if it is *.txt returned back to us because Filename Expansion failed then -f won't find a file called *.txt and [[ will return a fail.

Run cat

Having tested we have some real files, we could run cat again as we did originally:

% cat *.txt

but there's a hint of a race condition here in that someone could have removed the .txt files while we have been running.

We'd be better off using the list of filenames we received back from Filename Expansion which would leave cat complaining about missing files, which we know were there a moment ago when we expanded the wildcard, giving us a small clue as to what might be causing the problem in the first place.

There's also the reverse, something could add more .txt files while we are running: another *.txt expansion would pick them up. Ours is a very simplistic example but you're more likely to be processing these files including marking them as processed (perhaps by deleting them!) in which case operating on a fixed list of files which is generated once in your code is a much easier position to manage.

So we want our array expanded:

% cat "${txt_filenames[@]}"

or

% cat "$@"

Noting the quoting again.

Putting it all together

txt_filenames=( *.txt )
if [[ -f "${txt_filenames[0]}" ]] ; then
    cat "${txt_filenames[@]}"
fi

or

set -- *.txt
if [[ -f "$1" ]] ; then
    cat "$@"
fi