Work the Shell - When Is “Good Enough” Good Enough?

Shell scripting and obsessive-compulsive perfectionism is almost a contradction in terms.

Last month marked the end of my series about writing a Blackjack game as a shell script, and I don't know about you, but I had a good time with the development process and have even learned a bit more about the game itself. I received a number of fun e-mail messages from readers about the column, but I also received one that was most thought provoking.

The author criticized me for using less-than-optimal algorithms for things like my shuffle routine, for using poor scripting style and generally questioned how much I really knew about shell script programming in the first place.

Another Day, Another Flame

Having lived on the Internet for almost 30 years now, I'm quite familiar with flames and hostile e-mail, with people nit-picking, focusing on the molecules of the leaf without ever even knowing there's a forest ahead, but this message still got me thinking about the practice of scripting and of programming in general.

To quote a colleague of mine, Ken McCarthy, what's a better strategy, imperfect action or perfect inaction?

Although the Bourne Again Shell is remarkably capable and certainly has all the basic programmatic structures of more sophisticated programming languages, I think it's nonetheless fair to say that it's a lightweight, even throw-away programming environment. You don't write large, complex or mission-critical applications as shell scripts, do you?

That's how I have always approached shell script writing—basically as a fast prototyping environment. You want to know how many lines are in a file? Use something like:


lines=$(wc -l < $filename)

Is that the most elegant and efficient solution? Probably not. Indeed, if you're doing that to test whether the file has nonzero content, you should be using the test command instead, but here's my point: it doesn't really matter.

That's what I mean by imperfect action. It's far, far better to get going with your script, to build a sloppy prototype, to get it done, than to tune, clean up, tweak, rewrite and optimize until it's 3am two weeks after your deadline. That's perfect inaction, right? The zeal to wait and wait and tweak and prod until it really is perfect, by which time you've missed your deadlines and goals.

So, when I received the criticism from this reader that I hadn't chosen the best possible algorithms and wasn't using what he thought was an optimal scripting technique, I was glad to see how he thought things should be done. But I also was unsurprised, as one of the greatest challenges I believe facing software developers is learning that in many cases and situations “good enough” really is, well, good enough.

Writing a Shell Script Isn't Developing Software

Am I advocating that the next time you're writing the firmware for the in-flight controller on the Boeing 777 you should cut corners, skip testing and write crummy code so you can ship on time? Of course not. But you know what? If you're writing a testing framework cron script that will simply log the start of the test, invoke a series of MySQL queries and log the end of the test, well, yeah, in that case, relatively crummy code, code that works well enough, might just be exactly what's required.

I see this same perfectionist attitude with letters I occasionally receive from people who have bought my Wicked Cool Shell Scripts book. They haven't realized that writing a shell script is inherently an exercise in rough prototyping (hence the absence of sophisticated shell script development environments and testing frameworks), not a programming world where perfect code is the digital holy grail.

The “Not Good Enough” Developer Crowd

I also think that these rather snobbish software developers who worship elegance and dismiss software developed by people who are still stumbling their way through programming are doing a significant disservice to the world of computing.

Don't get me wrong, I appreciate an ingenious algorithm and snappy implementation when I read code, but most of the real innovations in software and applications come from the bubble gum and bailing wire crowd anyway, from the rough prototypes and the “barely beta” software that works well enough to demonstrate concepts and get the community to start experimenting anyway.

So I Should Be Writing Sloppy Code?

Some of you doubtless are a bit confused by this point in my column—puzzled that someone who is supposed to be teaching you tips and tricks of shell script programming is actually advocating sloppy coding and quick “throw-away” scripts. I ask instead why you're surprised at the idea that getting something out the door, solving the problem quickly and reasonably well, is often better than wasting—yes, wasting—time writing “the perfect script”?

This reminds me of a computer programming course I took many years ago at UC San Diego. Our challenge was to figure out the optimal sorting algorithm for a given situation and write a program that implemented it. Just about everyone in the class thrashed about, but I pulled out Knuth's Art of Computer Programming, picked out the algorithm he recommended, and typed it in, with an appropriate citation. I was penalized for “cheating” and had to argue adamantly that there was, in fact, no better way to learn how to choose the best sorting algorithm than to refer to the definitive work on the subject. Finally, the professor relented and gave me full credit.

Shell script programming is the same: shortcuts are always a good thing, efficiency is a measure of how fast you can solve a problem, and although tuning and tweaking can be rewarding as an intellectual exercise, 90% of the time it just doesn't matter at the end of the day.

Think about that. And ask yourself how you're working toward imperfect action rather than being trapped trying to achieve perfect inaction.

Next month, we'll go back to the nuts and bolts of shell scripting. But, please, don't expect me to write perfect little scripts or use the absolute best algorithm in the world for a given task. Indeed, sometimes my code will be inefficient, will spawn more subshells or child processes than entirely necessary, or might even have unnecessary loops and conditionals. Maybe, just maybe, that's okay?

Dave Taylor is a 26-year veteran of UNIX, creator of The Elm Mail System, and most recently author of both the best-selling Wicked Cool Shell Scripts and Teach Yourself Unix in 24 Hours, among his 16 technical books. His main Web site is at www.intuitive.com.

______________________

Dave Taylor has been hacking shell scripts for over thirty years. Really. He's the author of the popular "Wicked Cool Shell Scripts" and can be found on Twitter as @DaveTaylor and more generally at www.DaveTaylorOnline.com.

Comments

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

While shell scripts are

A. T. Young's picture

While shell scripts are indeed great for fast prototyping, they're equally useful for mission-critical production code. Look at the scripts that installed your system, and the ones that bring everything up each time you boot.

In addition, even large, complex applications can be written as shell scripts -- especially with the fast hardware we have these days. A few years ago, I got tired of continually re-writing a big nonlinear damped-least-squares program that I'd been using for decades, every time a new problem came up. I noticed that the changes needed were so mechanical that I could let the machine do that work for me. So I now have a 2500-line shell script that looks at the data, infers its format, asks the user a few questions (like what the model is to be fitted), and collects the necessary pieces to form the source code for the program, compiles it, and then executes the program. Ten years ago I'd never have dreamed of putting a compilation inside a shell script, but in fact it all proceeds smoothly, and you hardly notice the little momentary pause while the compilation takes place.

While lots of shell scripts are indeed throwaways that need only to be barely good enough for one-time use, a surprising number turn out to be things you find yourself using again and again -- often in spite of your original intentions. So I've found it pays, in the long run, to take the time to add enough comments to document the code for the time a year from now when I find myself looking at it cold again.

I agree that optimizations are things you only do when you find the thing runs too slowly, though. Often, this is best done by finding the slow part of the script and writing just that one bit as a compiled program in your favorite language, leaving the rest of the procedure as a shell script.

Good enough

martyskitch's picture

Great article and as I tell my kids when they are doing homework "Prefection is the enemy of good enough".

Webinar
One Click, Universal Protection: Implementing Centralized Security Policies on Linux Systems

As Linux continues to play an ever increasing role in corporate data centers and institutions, ensuring the integrity and protection of these systems must be a priority. With 60% of the world's websites and an increasing share of organization's mission-critical workloads running on Linux, failing to stop malware and other advanced threats on Linux can increasingly impact an organization's reputation and bottom line.

Learn More

Sponsored by Bit9

Webinar
Linux Backup and Recovery Webinar

Most companies incorporate backup procedures for critical data, which can be restored quickly if a loss occurs. However, fewer companies are prepared for catastrophic system failures, in which they lose all data, the entire operating system, applications, settings, patches and more, reducing their system(s) to “bare metal.” After all, before data can be restored to a system, there must be a system to restore it to.

In this one hour webinar, learn how to enhance your existing backup strategies for better disaster recovery preparedness using Storix System Backup Administrator (SBAdmin), a highly flexible bare-metal recovery solution for UNIX and Linux systems.

Learn More

Sponsored by Storix