Moving Up The Rings

Many things have rings: mobile phones have incredibly annoying ones, jewelers have incredibly expensive ones, and Hell — at least according to Dante — has incredibly detailed ones. For the past three years, thanks to a government contractor called Coverity, Open Source has rung as well.

Coverity began contracting with the Federal Government in 2006, after the Department of Homeland Security began to wonder about the quality of the Open Source offerings being used by fellow feds. The company builds code-analyzing tools aimed at finding vulnerabilities and other hiccups in the programming process, and thanks to the government's inquisitive nature, those tools have been turned on Open Source for the past three years.

The company's system is unlike the more traditional find, report, and fix approach where developers and users running the applications identify problem areas as they present themselves and correct as necessary. Coverity uses static analysis, which performs its review without running the software. The method doesn't identify certain types of issues, as Forrester Research's Jeffrey Hammond pointed out to IDG News: "Static analysis [tools] won't tell you that your business process is working correctly...but they will tell you that the code itself is technically solid."

According to Hammond, static analysis looks primarily for poor programming — "structural 'anti-patterns' in code" — identifying "more exotic" issues including parallel code execution, as well as more common problems like buffer overflows and memory leaks. The process identifies whether code "follows the kind of programming best practices you'd expect to see from code that has gone through a proper code review."

The analysis process, which relies on voluntary submission of code for review, uses a rung system to classify how far the project has progressed in correcting the problems discovered in during analysis. Coverity has assigned four projects — OpenPAM, Ruby, Samba, and tor — to Rung 3, the final step on the bug-squashing ladder.

Coverity reports that 280 projects have submitted code for review, representing over sixty million lines of code. More than 11,200 bugs have been eliminated, with coders from some 180 projects working to scan submitted code. The program has dramatically decreased what Coverity calls "defect density," down sixteen percent in three years.