The U.S. Software Industry and Software Quality: Another Detroit in the Making?
"The reason we come up with new versions is not to fix bugs. It's absolutely not. It's the stupidest reason to buy a new version that I've ever heard.... And so, in so sense, is [software] stability a reason to move to a new version. It's never a reason. You won't get a single person to say they'd buy a new version because of bugs."
--Bill Gates, CEO of Microsoft Corporation, in an interview with reporter Klaus Brunnstein (Focus, November 1995)
Do computer users care about quality? Linux advocates hope so, because it's unquestionably the case that open-source development methods are capable of producing some very fine software indeed. As open-source guru Eric Raymond points out, the nature of open-source development - such as the wide-open availability of the underlying source code, the ongoing testing of code in real-world settings, the frequent release cycles - can produce code that's remarkably free from programming errors. The sheer number of developers helps, too; as Linus Torvalds puts it, "With enough eyeballs, all bugs are shallow."
To be sure, not every program developed with open-source methods is as beautifully crafted as the Linux kernel, but there's no disputing the fact that open-source development can indeed produce software of exceptional quality. If quality matters, Linux ought to have an edge over its commercial competitors. According to one estimate by a Microsoft internal (see Minasi 2000:255), the firm's products typically contain an average of 14 to 17 errors per 1,000 lines of code - a level of quality that can be described as mediocre. But people keep buying Microsoft products. Vendor executives, Microsoft's among them, look at their profits and ask why they should bother improving their firms' software. Sure, they admit, it's possible to produce software of space-shuttle quality, but doing so is very expensive. Maybe that level of quality is needed in life-critical systems, such as medical software, but who needs a quality word processor? Consumers don't care, they conclude, and so they keep putting out products that are "good enough".
They're wrong. Dead wrong. Consumers have been putting up with bug-ridden software for one simple reason: They don't realize there is an alternative. And once they find out, commercial software vendors are going to lose a big slice of their business. Where's my evidence for this claim? History. I'm sure you've heard the famous Santayana quote: "Those who do not remember the past are condemned to repeat it." (No, that's not a typo; it's Santayana the philosopher, not Santana the guitarist.) If you're looking for an example, I've got a doozy for you. According to Mark Minasi, author of a very fine book entitled The Software Conspiracy (McGraw-Hill, 2000), the U.S. commercial software industry is making exactly the same mistake that U.S. auto makers once made, and the results could prove catastrophic to the U.S. economy.
Flash back to the 1950s, and take a look at the average new car produced by one of Detroit's "Big Three" auto makers (GM, Ford, and Chrysler). You'd see lots of cool features: big, gutsy V-8 engines, flashy chrome bumpers, and (in 1957, anyway) fins that made the cars look like low-flying rockets.
If you owned one of these monsters, though, you'd discover another, less-appealing characteristic: shoddiness. The cars were riddled with defects and needed frequent repairs. They weren't safe, either, and they were murder on the environment. Instead of improving their products and making them safer and less polluting, the Big Three auto makers went to work on the politicians. They did everything they could to ward off legislation to give consumers protections against lemons. They opposed air bags. They tried to fight off pollution standards. In today's markedly more corrupt political environment, they probably would have succeeded.
They also went to work on consumers. Money that could have gone into improving their products, as well as making them safer and less polluting, went into advertising and marketing instead. The goal? Get consumers back into the showroom every two or three years to buy a new car with new, up-to-date styling. Under the hood, of course, they got the same old junk.
Call it shortsightedness, if you'd like, or just plain greed, but the Big Three auto makers couldn't see a financial incentive for improving their products. So they didn't. They knew the cars were junk. They knew they were unsafe. Sure, every once in a while, they had little twinges of conscience - such as when an auto executive's kid was killed in a fiery crash, one that could been prevented had the company paid more attention to safety. They felt terrible for a few days. (You can read the whole, sick story in J. Patrick Wright's On a Clear Day You Can See General Motors, published in 1979.) But all such concerns were sacrificed to the Bottom Line. When challenged to defend their low-quality cars, the auto makers complained that the cost of building quality automobiles was simply too high; it could be done, but you'd pay at least half again as much for that shiny new Chevy. Consumers were content with the low quality/low price tradeoff, the auto makers believed. Consumers are buying the cars, they pointed out. The auto makers were raking in fabulous profits, and making a fantastic contribution to the economy.
In fact, consumers weren't content with the cars (or the dealers, but that's another story). Still, complaining didn't get them anywhere, and for one simple reason: there wasn't any competition. If U.S. cars were shoddy, they looked like the space shuttle next to British cars, which (lamentably) lacked the capital to do anything about their endemic quality problems. Sure, there were some little Japanese companies that were making funny-looking, inexpensive cars, but these companies weren't a threat to Detroit, the auto makers believed. Japanese car makers didn't know anything about marketing and style, and that's what sells cars in the U.S.
You probably know the rest of the story. For years, U.S. industrial quality guru W. Edwards Deming tried to convince Detroit that it was possible to make high-quality products, and in addition, it's not much more expensive to do so, as long as you design the quality into the product at the beginning of production instead of trying to fix the problems at the end. But Demming's words fell on deaf ears - except in Japan.
Japanese car makers took Demming's teachings to heart, and they started making some exceptionally fine automobiles. What's more, they were cheap. The result? Japanese auto makers grabbed nearly a third of the U.S. market and most of the international market. As a result, thanks to mounting Japanese automobile exports and the collapse of the U.S. auto industry overseas, the U.S. was plunged into the ranks of the world's debtor nations.
Detroit's story should be clearly understood by everyone who wishes to grasp the significance of shortsighted, bottom-line thinking in corporations besotted by too much testosterone. Sure, you make money. In reality, though, you're doing so only by mortgaging your country's future. You're pushing for laws that, if passed, would have rolled consumer and environmental protection back to the Dark Ages. You're creating lasting ill will in a market that despises your products, and looks desperately for an alternative. And if you fail to keep your competitors out of the market, you go down - and you take a huge slice of the economy with you. But who cares? Your kids and grandkids will pay, not you.
Today's commercial software packages have much in common with shoddy U.S. automobiles of the 1950s and 1960s, according to the software industry's critics. It's basically the same formula: put out shoddy products, and use high-pressure marketing to keep consumers focused on new software versions that offer glitzy new features. In reality, you're hoodwinking people into buying the same defective product over and over again, but hey - you make tons of money. And who cares about quality, anyway? Sure, industry executives concede, we could reduce the number of bugs in our products, they say, but only by raising the price of our products by 50 percent or more - and consumers won't stand for it. Quality? We'll give you "good enough" quality, and that's all you're going to get.
It's incredibly cavalier of these companies to say that quality isn't needed in products such as word processors, spreadsheet programs and the rest. People have lost jobs, flunked classes, and contemplated jumping off bridges after software glitches destroyed work that was critical to them. And these very same products are finding their way into virtually every aspect of life, including situations in which human life and limb could very well be at stake if the e-mail doesn't get through. Shoddy, bug-ridden software isn't safe to use under any condition, and these companies know it. My evidence? Instead of improving their products, commercial software vendors are busily trying to rewrite U.S. and international law to shield themselves from the consequences of their corporate negligence. In the U.S., Microsoft has taken the lead in pushing for the passage of UCITA, a state-level legislative act that has been opposed by every consumer rights organization that has ever examined the issue, as well as by 23 U.S. Attorneys General and computing professional organizations, who correctly describe the legislation as a major setback not only for consumers, but also for public safety.
Sounds like the Detroit game all over again, doesn't it? But wait: there's more. Inspired by Demming's writings, software development expert Watts Humphrey - an ex-IBM executive who is now affiliated with Carnegie-Mellon University's Software Engineering Institute (SEI) - developed a version of Demming's work for the software industry. And guess what? U.S. software vendors aren't listening to Humphrey. One reason they're not listening is that they're too busy jeering Humphrey and ridiculing his work, which is exactly what U.S. auto-industry executives did to Demming in the 1950s. (Demming eventually gave up and moved to Japan.)
What's Humphrey saying? It's simple: software companies can make high-quality products, and what's more, doing so isn't expensive. Humphrey's work has evolved into the Capability Maturity Model (CMM), which shows software developers how to build quality in from the get-go. It also provides a way of ranking a company's commitment to quality. At Level 1, companies aren't doing much of anything about quality. At Level 5, they're up to the Toyota level: they're building quality consciousness into everything they do, and they're constantly refining and improving their processes.
What's more, CMM works. Using CMM-like methods, telecommunications giant US West Technologies was able to reduce service outages by 79 percent, slice billing costs by $30 million, and reduce service order errors by 50 percent. There's an upfront investment required, to be sure, but it pays off in the long run. In 1990, the cost of ensuring quality at Raytheon Electronics Systems ate up nearly two-thirds of all software development costs. Thanks to CMM, Raytheon is putting out even better software, but the cost of assuring this quality has fallen below 10 percent of software development budgets. And what about bugs? Based in Chennai, India, a contract software developer called Advanced Information Services (IAS) - one of the few CMM Level 5 companies in existence - is cranking out code with only 0.05 defects per thousand lines of code. That's better than the space shuttle's software. This level of achievement isn't putting IAS out of business - far from it: their profits have doubled. On average, companies that adopt CMM realize a fivefold return on their investment.
Who's listening to Humphrey? CMM critics affiliated with Microsoft charge that CMM creates an unwieldy bureaucracy that forestalls the kind of brilliant innovation that's leading the software industry. Give me a break! If Microsoft supposedly exemplifies the type of organization that would be "paralyzed" by CMM to the point that it couldn't innovate, we might all be much better off. As near as I can tell, the lion's share of Microsoft products that could be termed "innovative" in some sense - MS-DOS, Windows, FrontPage and others - either originated outside the company, were based on ideas that were developed outside the company, or were acquired by purchasing an outside company. Microsoft's innovations seem limited to figuring out new ways of introducing dysfunctional extensions to prevailing standards for no other reason than the firm's desire to put its competitors out of business.
So who is listening? As of this writing, only 19 software companies are certified at Level 5, and 13 of them are in India. That's right: India. If you think India is a backward country that couldn't possibly compete in the high-tech sweepstakes, you'd better think again, because Indian software companies are putting out some of the best software in the world. Near Bangalore, India, a CMM-driven, Level 5 shop is turning out software with 0.03 defects per thousand lines of code. Right now in India, there's a replay of exactly the same process that energized the Japanese automobile industry thirty years ago. They see the opportunity. They have the talent. They know they can create world-class software. They're doing it right now.
When asked whether Indian software firms pose a threat to their near stranglehold on the consumer software market, U.S. software executives laugh. They point out that these silly foreign companies don't know anything about style or marketing; there's no way they could make it in the U.S. market. Now where have we heard that before?
First, buy ten copies of Mark Minasi's exceptionally fine book, The Software Conspiracy: Why Software Companies Put Out Faulty Products, How They Can Hurt You, and What You Can Do About It (McGraw-Hill, 2000) and give a copy to everyone you know who's in a position to influence software purchasing decisions. Mail them a copy of this article, too. My major criticism of Mark's book is that he doesn't weave open-source software in general, and Linux in particular, into the picture. I believe Linux is exposing the need and the thirst for high-quality software, and that the high quality of Linux and other open-source programs is in large measure responsible for its rapid growth and acceptance.
Second, get the word out about UCITA. Visit http://www.badsoftware.com and the Consumer Project on Technology home page (http://www.cptech.org). If you're in a U.S. state where UCITA next comes up for ratification, organize street protests, write letters to legislators, get the word out, and fight it! There's still time to defeat this monster, but we have got to get organized NOW!
Third, join the battle to promote software engineering, high-quality software, and responsible behavior by software vendors. Read Watts Humphrey's comments on software quality at http://www.2bguide.com/docs/whsq.html and then visit the Software Engineering Institute (SEI) at Carnegie-Mellon University, with which Humphreys is affiliated.
Fourth, convince any organization with which you are affiliated - your school, your company, the non-profit organizations where you volunteer - that purchasing commercial vendors' products is aiding and abetting a process that is exposing the public to unwarranted risk, generating legislation that is harmful to public welfare, and retarding the progress of technology. Show them Linux, help them install it, and invite them to consider what people are slowly but surely learning: you don't have to put up with shoddy software.
Bryan Pfaffenberger is a professor in the new Media Studies program at the University of Virginia, where he will teach two courses next fall (Media Studies 317: Intellectual Property and Digital Media, and Media Studies 110: Information Technology and Digital Media). He lives in Charlottesville, Virginia.