On the Xtreme in Open-Source Software Development

What do Xtreme Programming and open source have in common and what can they learn from each other?

Recently, software developers have been offered a plethora of remedies that seek to address inherent inadequacies in and observed problems with traditional software development methodologies. More often than not, however, these come bundled with their own variants of said inadequacies and problems. As a result, at least as far as software developers are concerned, these remedies basically replace an old devil with a new one.

Not all of the remedies fall into this category, however. One particular approach that has seen some well-deserved success is Xtreme programming. Xtreme programming (XP), with due credit to its founding fathers, bases its processes and practices on elements of simple and sound common sense. XP focuses on the things that matter in the development of software. Whole volumes are available on XP that discuss what really matters in software development, and we don't dwell on it here because this article is not about XP. This article is about open-source software development. Given the ideal of better software that remains the guiding principle of the open-source world, open source inevitably crosses paths with XP, in philosophy and in practice. In the rest of this piece, we examine some Xtreme influences on open-source software development--what they are and what they can be.

The Customer

XP recognizes that the single most important factor in software development is the customer. Most key practices in XP revolve around communication with the customer, listening to his requirements and responding to his feedback. XP even advises that you have a representative of the customer available at the development site throughout the development life cycle. Open-source projects have a quirky and largely unintentional similarity to this approach. Open-source projects, almost without exception, take birth as tools to serve some personal need of the developer, who eventually comes to believe the tool may benefit the rest of mankind and publishes it. Others then try it, improve it, hack it, rewrite it or supplement it to address some missing feature or fix a bug. So, at least in the beginning stages of the development, the developers are the customers. Can there be a more idyllic arrangement? And when the package actually is released to the community, rich feedback in terms of reviews, bug reports, usage experiences and comparisons show up on the ubiquitous mailing lists. All in all, the customer factor in open-source development is handled in a manner that even XP can learn from.

Releases

Most conventional software development methodologies work on a single release schedule. The development life cycle spans years and essentially is a black box from the point-of-view of the customer, who has to step out of the way and await the day the developers transform his requirements into usable software. This approach offers no scope for change, for feedback from the customer on what is being churned out in the development process or for a cut-and-run.

For customers who need some degree of control over the development process, XP often is the answer. XP proposes an incremental release schedule that incorporates a finite subset of product features (sorted by business value) in each release. Customers feed requirement changes as well as their comments on the current release into a system that adopts these into the release cycle. More importantly, each release is thoroughly tested and the customer is assured of flawless execution within the scope of the limited feature set. Iterative development of this sort is the norm in open-source development. In fact, the open-source SDLC has been referred to disparagingly as release-first.

To be fair, some of this criticism is deserved. Open source-package releases often are buggy and full of untested code, thanks to authors who have come to depend on bug-reports and feedback from users to such an extent that they do not perform elementary sanity-checks on their code before publishing it. To be fair some more, to the developers this time, it must be mentioned that such releases are tagged appropriately and advertised as unstable. However, that's no excuse. One lesson that open-source developers can pick up from XP is to ensure that any release that goes out of the stable is thoroughly tested in scenarios the developers are familiar with. Feedback from the field then can be relied on for scenarios untestable in the development setup or those that have been overlooked.

In the same vein, XP assures its customers of strict adherence to a pre-planned release schedule. Although the scope of each release may vary in consultation with the customer, the deadlines remain immovable. On the contrary, the release schedule for open-source projects is, to say the least, haphazard. Planning is sometimes based on a TO-DO list but more often is governed by essential bug fixes and the triviality factor of feature additions. Because most development endeavors in the open-source world happen in the off-hours, when the developers are not at their regular day jobs, we sometimes see releases that follow each other by scarcely a day only to be followed by lulls that may span years. Exceptions to this are large projects with roadmaps that sketch the broad outline for a release schedule.

______________________

Comments

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

Re: On the Xtreme in Open-Source Software Development

Anonymous's picture

I find discussions of the overlap in these two areas to be very interesting, however, I thought there were some fairly obvious omissions/errors, particularly with respect to XP.

To me, "collaboration" is the most important similarity in these two topics, but this is at a deeper level in both XP and Free / OSS. More of a shared value than a particular practice. On a similar note, both share "self-organizing" concepts.

How can you mention XP, without at least mentioning that it is an Agile Process (see http://www.agilealliance.org)

Also, a significant number of XP tools are being developed as Free / OSS projects. These are examples where the two types of projects have become "one and the same".

Take most of the xUnit family of Automated Unit Test tools.
Also, consider fitnesse, and some other Automated Acceptance Test frameworks starting to come out. Not to mention, a number of efforts at developing refactoring tools.

One last minor point: It's called "eXtreme Programming", not "Xtreme Programming". See http://c2.com/cgi/wiki?ExtremeProgramming.

Cheers,
Jason Nocks

Re: On the Xtreme in Open-Source Software Development

vinodv's picture

I agree with most of the points you make esp. the one about collaboration. It certainly merited some more elaboration. The 'self-organizing' aspect of both these disciplines (more so of OSS, methinks) is an interesting idea and one that I'm afraid didn't occur to me when I wrote this. Thank you for bringing it up.

As for your other points, the focus of the article was to compare/evaluate processes and practices in the 2 methodologies, highlighting similarities in intent and possible differences in the actual implementation. In the light of that, the fact that most XP tools are OSS projects or XP (being a test-driven development methodology) has spawned several AUT toolsets hardly bears mention.

Sure I know it is called Extreme Programming (http://www.extremeprogramming.org) :).
'Xtreme' is just more uber ;)

vinod

Re: On the Xtreme in Open-Source Software Development

Kishoreks's picture

Very informative article. Offers lot of food for thought. I find myself agreeing to lot of things. Also, I find myself dis-agreeing with some. Since it is redundant to state what one agrees upon, I'll just state my disagreements:

1. The section "Refactoring" does not bring any XP facets onto the table: none of the XP ways for Refactoring are described. Hence, the reader is left wondering "what IS the influence of XP, as related to Refactoring, in the Open Source Software Development"?

This section also does not say much about HOW Open Source Software Development projects are dealing with this. It seems to suggest what could be done as against what IS being done. At least here, examples of real projects that have dealt with Refactoring is warranted.

2. A system metaphor is expected to go beyond just the shared code base. Any design details, requirement discussions etc should use certain "terms" and "definitions" that conveys the same thing to everybody. This aspect needs to be stressed. Also, it will be useful to the reader if the various "items" that constitute system metaphor in XP were mentioned in the passing, especially as applicable to Open Source Software development (from the writeup, only Coding Standards seems to be the "implied" system metaphor).

3. Under Section "Formalism", you have said:
"... But the denizens of the open-source world are well-versed in the art of browsing code to understand design and have never needed reams of paper describing it."

I think the above is, to some extent, "putting the cart before the horse". When we view software, in your own words, as "something that address what the customer wants his software to do (requirements) and how the developer satisfies these needs (design and implementation)", and we by-pass describing the design ("go read the code", "the design is in my head", "read the 1000 emails on the mailing archive. Design is spread-across and buried in those emails" are typical outbursts), we make a grave mistake -- the final code is usually a "short, not-so-easy-to-absorb, formal description", especially of the "various considerations" that led to it which the reader totally misses! The Open Source Developer does have some design in mind, and that changes over time (which is perfectly reasonable), and while it is shaping and re-shaping the implementation and is being cast and re-cast in concrete, he/she is too lazy to describe what is shaping and re-shaping it.

In the same section, you have later said:
"To their credit, open-source developers have been quick to recognize this. Most big open-source development efforts today are backed with the necessary design descriptions."

The above is "let's now put the horse before the cart" and giving credit to some thing that should have obviously been there in the first place!

Re: On the Xtreme in Open-Source Software Development

Anonymous's picture

Hello VV,

Your article was very good but you left out what I feel is the biggest XP contribution: automated unit tests. I wouldn't even try refactoring without unit tests to validate the results.

Sincerely,
triso@cogeco.ca

Re: On the Xtreme in Open-Source Software Development

vinodv's picture

Hi,

In the article, I have covered only those facets of XP that (I believe) _already_ have some reasonable equivalent in the Open Source development model. While I agree that the concept of automated Unit tests is a significant contribution from XP's stables, I feel that there is no real equivalent (in-place) in the Open Source world. The article wasn't meant entirely as a session on 'Lessons OpenSource developers can pick up from XP'. Rather, it was a comparitive evaluation of some key practices in both models that were similar in intent (but perhaps were better implemented in one model).

Hope that explains the omission.
-- Vinod Vijayarajan

Re: On the Xtreme in Open-Source Software Development

Anonymous's picture

According to the XP litterature refactoring isn't possible without automated tests. Without the tests what really goes on is something else, whether you call it code cleanup, redesign, whatever. That's refactoring according to XP anyway. From that point of view, I hear the word refactoring being misused a lot.

Just a thought.

Cheerio,
Malte Tancred

Re: On the Xtreme in Open-Source Software Development

Anonymous's picture

The section on refactoring in this article seemed to completely miss the point. As you mentioned, the word refactoring is misused a lot. The refactoring section in this article is just another instance of such. "Refactoring is not rewriting." See http://c2.com/cgi/wiki?RewritingIsNotRefactoring.

Cheers,
Jason Nocks