Work the Shell - Compact Code and Cron Contraptions
This month, I thought I'd take another sidetrack. (You knew that entrepreneurs all have ADD, right?) So, it should be no surprise that to me, the fastest way from point A to point B is, um, what were we talking about?
Reader Peter Anderson sent in a code snippet that offers up a considerably shorter way to convert a really big byte count into kilobytes, megabytes and gigabytes than the one I shared in my December 2006 column.
His question: “Why so much extra code?”
His snippet of code to do this takes advantage of the built-in math capabilities of the Bash shell:
value=$1 ((kilo=value/1024)) ((mega=kilo/1024)) ((giga=mega/1024)) echo $value bytes = $kilo Kb, $mega Mb and $giga Gb
Peter, you're right. This is a succinct way of solving this problem, and it's clear that a shell function to convert, say, bytes into megabytes easily can be produced as a one-liner. Thanks!
As I've said in the past, I don't always write the most concise code in the world, but my goal with this column is to write maintainable code and to get that prototype out the door and be ready to go to the next thing as fast as possible. That practice isn't always compatible with the quest for elegance and perfection in the coding world, to say the least!
On an admin mailing list, I bumped into an interesting question that makes for a perfect second part to this column—a simple script that's really just a one-line invocation, but because it involves the cron facility, becomes worth our time.
The question: “I need to run a cron job that looks in a certain directory at the top of every hour and deletes any file that is more than one hour old.”
Generally, this is a job for the powerful find command, and on first glance, it can be solved simply by using an hourly cron invocation of the correct find command.
For neophyte admins, however, there are two huge steps involved that can be overwhelming: figuring out how to add a new cron job and figuring out the correct predicates for find to accomplish what they seek.
Let's start with find. A good place to learn more about find, of course, is the man page ( man find), wherein you'll see there are three timestamps that find can examine. ctime is the last changed time, mtime is the last modified time and atime is the last accessed time. None of them, however, are creation time, so if a file was created 90 minutes ago but touched or changed eight minutes ago, all three will report eight minutes, not 90. That's probably not a huge problem, but it's worth realizing as a very typical compromise required to get this admin script working properly.
For the sake of simplicity, I'll actually change this example to deleting files that haven't been accessed in the last 60 minutes, not worrying about how much earlier they might have been created. For this task, I need ctime.
find has this baffling syntax of +x, x and -x for specifying 60 minutes, and it would read as “more than x”, “exactly x” and “less than x”, respectively. If we use the sequence -ctime -60, we'll get exactly the opposite of what we want; we'll get files that have been changed in the last 60 minutes.
Or is that what we are specifying? Without a unit indicated, the default time unit is really days, so -60 is actually files that have been changed in the last 60 days—not what we want!
To specify minutes, we want to use cmin rather than ctime (I told you find was confusing). Here's how that might look:
find . -cmin +60
The above also matches directories, however; so another predicate we'll want to add is one that constrains the results only to files:
(type d is only directories, and so forth).
But, that's not exactly right either, because we probably want to ensure that we only ever go one level deeper instead of spending a lot of time traversing a complex file tree. This is done with the little-used maxdepth parameter, which is described as “True if the depth of the current file into the tree is less than or equal to n.” Now, let's put this all together:
find . -cmin +60 -type f -maxdepth 1
See how that all fits together?
Now, the last part of this requirement is actually to delete the matching file or files, and I have to admit that this gives me some cause for anxiety, because if you make even the slightest mistake with the find command, you can end up deleting tons of files you didn't want removed—not good. So, rather than just use -delete, I suggest you use -print, and for a day or so, let it run and have cron automatically e-mail the resulting report to you.
Dave Taylor has been hacking shell scripts for over thirty years. Really. He's the author of the popular "Wicked Cool Shell Scripts" and can be found on Twitter as @DaveTaylor and more generally at www.DaveTaylorOnline.com.
Practical Task Scheduling Deployment
July 20, 2016 12:00 pm CDT
One of the best things about the UNIX environment (aside from being stable and efficient) is the vast array of software tools available to help you do your job. Traditionally, a UNIX tool does only one thing, but does that one thing very well. For example, grep is very easy to use and can search vast amounts of data quickly. The find tool can find a particular file or files based on all kinds of criteria. It's pretty easy to string these tools together to build even more powerful tools, such as a tool that finds all of the .log files in the /home directory and searches each one for a particular entry. This erector-set mentality allows UNIX system administrators to seem to always have the right tool for the job.
Cron traditionally has been considered another such a tool for job scheduling, but is it enough? This webinar considers that very question. The first part builds on a previous Geek Guide, Beyond Cron, and briefly describes how to know when it might be time to consider upgrading your job scheduling infrastructure. The second part presents an actual planning and implementation framework.
Join Linux Journal's Mike Diehl and Pat Cameron of Help Systems.
Free to Linux Journal readers.Register Now!
- SUSE LLC's SUSE Manager
- My +1 Sword of Productivity
- Murat Yener and Onur Dundar's Expert Android Studio (Wrox)
- Non-Linux FOSS: Caffeine!
- Managing Linux Using Puppet
- Doing for User Space What We Did for Kernel Space
- Tech Tip: Really Simple HTTP Server with Python
- Parsing an RSS News Feed with a Bash Script
- SuperTuxKart 0.9.2 Released
- Rogue Wave Software's Zend Server
With all the industry talk about the benefits of Linux on Power and all the performance advantages offered by its open architecture, you may be considering a move in that direction. If you are thinking about analytics, big data and cloud computing, you would be right to evaluate Power. The idea of using commodity x86 hardware and replacing it every three years is an outdated cost model. It doesn’t consider the total cost of ownership, and it doesn’t consider the advantage of real processing power, high-availability and multithreading like a demon.
This ebook takes a look at some of the practical applications of the Linux on Power platform and ways you might bring all the performance power of this open architecture to bear for your organization. There are no smoke and mirrors here—just hard, cold, empirical evidence provided by independent sources. I also consider some innovative ways Linux on Power will be used in the future.Get the Guide