Tech Tips with Gnull and Voyd

—Brad Hall

I knowed somebody was gonna get to this problem sooner or later. You get too many inodes on your system, and you're asking for another heap of trouble. This tells you how to find out and fix it. —Chester

One of the most common tasks of a system administrator is storage management. When you're faced with a full or almost full filesystem, it's good to have a few tools at your disposal to help figure out “where” the hog is.

Searching for space hogs is very easy. With one simple command you can total up the contents of every directory in a tree (/usr in this case) and see which are the largest:

du -k /usr/ |sort -n |tail -30

167156  /usr/include
168960  /usr/share/icons
173972  /usr/share/texmf
244332  /usr/bin
263144  /usr/lib/openoffice.org2.0
265492  /usr/share/doc
344536  /usr/share/locale
1223992 /usr/lib
1959412 /usr/share
4159996 /usr/

As you can see, /usr/share/ and /usr/lib/ are pretty big, and you can drill down further by going up the list.

A somewhat rarer situation is running out of inodes in a filesystem. In this case, you will see available space, but the system will be unable to write new files because it has run out of inodes. To find inode hogs, use this quick Perl script named inodu:

#!/usr/bin/perl -w

my $start=$ARGV[0];

foreach $object (`find '$start'`){
        my @parts=split(/\//,$object);
        while(pop(@parts)  ){
                my $object = join('/',@parts);
                $object =~ s/\/+/\//g;
                $object2qty{$object}++;
        }
}

foreach $object (sort { $object2qty{$a} <=> $object2qty{$b} }
                         keys %object2qty){
        print $object2qty{$object} . "\t${object}\n";

}

This will total up the number of filesystem objects in each directory and supply an output much like the previous example. Use it like this:

cd /usr

./inodu .

10420   ./include
10973   ./share/texmf
12012   ./share/man
13207   ./share/doc
14953   ./share/icons
16481   ./src/kernels
17201   ./src
22982   ./lib
105527  ./share
174270  .
174271

As you can see, again share and lib are the inode hogs using more than 100,000 inodes!

If you find yourself in any of these situations, there are a number of ways to create more free space or inodes. First, look for log files that can be purged, moved or compressed. Ask users to clean up their home directories. Remove any unnecessary software. If you are using Linux LVM and ext3fs, you can expand the filesystem using lvresize and resize2fs to grow a filesystem. This creates more free space and inodes, but only if you have free space in your volume group. If you have free disk space, you can create a new partition (for, say, your /var tree), move the files to that partition and mount it as /var. As a last resort, you can move files and directories and use symlinks so the old path still works. I say “last resort” because this method can get out of hand very quickly and can make things very confusing.

—Matthew Hoskins

Linux Journal pays $100 US for any tech tips we publish. Send your tips with your contact information to techtips@linuxjournal.com.