[Updated Feb. 17, 2003] Reasoning Inc. announced that it is making available a study that reports that the TCP/IP protocol stack implementation in version 2.4.19 of the open source Linux kernel has fewer defects than the TCP/IP protocol stacks of several commercial equivalents. The study by Reasoning is the first published comparison of code defect assessment of an open source software application versus equivalent commercial software applications. The white paper can be obtained free-of-charge, here (PDF download).

Reasoning's study compared six implementations of TCP/IP, the fundamental protocols underlying the Internet. Besides the Linux kernel, three of the implementations were part of commercial general-purpose operating systems, and two were embedded in commercial telecommunications equipment. Reasoning found 8 defects in 81,852 lines of Linux kernel source code – the fewest number of defects of the various implementations of TCP/IP inspected by Reasoning as part of its study.

Reasoning's code inspection service is based on a combination of proprietary technology and repeatable process. The results are objective and comparable across operating systems, vendors, and coding styles, which has enabled Reasoning to build the world's largest database of code quality metrics.

“Proponents of open source software have long argued that open source software is as or more robust than commercial software,” said Reasoning President & CEO Scott Trappe. “Having inspected hundreds of commercial applications, Reasoning can provide direct evidence to support their claim. We are pleased to be able to provide scientific data, adding to the intelligent discourse on best practices within software development and how to produce better quality software.”

Studies have long shown that source code inspection is the most effective technique for improving software quality. New programming methodologies such as Extreme Programming codify this by having two programmers write each line of code together. Open Source makes the source code to applications available to anyone. Hundreds or even thousands of programmers can review each other's work, identify problems and suggest fixes. Trappe notes that the open source model encourages several behaviors that are uncommon in the development of commercial code:

• First, many users don't just report bugs, as they would do with commercial software, but actually track them down to their root causes and fix them.
• Second, many developers are reviewing each other's code, if only because it is important to understand code before it can be changed or extended. It has long been known that peer reviewing is the most effective way to find defects.
• Third, the open source model seems to encourage a meritocracy, in which programmers organize themselves around a project based on their contributions. The most effective programmers write the most crucial code, review the contributions of others, and decide which of these contributions make it into the next release.
• Fourth, open source projects don't face the same type of resource and time pressures that commercial projects do. Open source projects are rarely developed against a fixed timeline, affording more opportunity for peer review and extensive beta testing before “release.”

The results of Reasoning's comparison of six implementations of TCP/IP are in a white paper, “How Open-Source and Commercial Software Compare”.

Dr Jasper Kamperman, a senior software inspection expert at Reasoning, will present the findings at Real World Linux in Toronto, April 29.

 
This article was originally published on LinuxDevices.com and has been donated to the open source community by QuinStreet Inc. Please visit LinuxToday.com for up-to-date news and articles about Linux and open source.