On the Linux Lunacy Geek Cruise last month, I had the pleasure of getting some very educational hang-time with 2.6 kernel maintainer Andrew Morton and filesystem guru Ted Ts'o. I came away not only with enormous respect for Andrew, Ted and the kernel crew, but with an even deeper realization that Linux is not an OS in the sense we've been taught to think about one by the commercial marketplace.

Meaning, Linux isn't a product.

Linux is a project, of course. Specifically, it's an open-source development project. But that's a technical explanation, a noun compound. As a description, "project" doesn't convey the vast difference in nature between Linux and commercial platforms such as Microsoft's Windows and Apple's MacOS. Even when Linux is packaged into a distro, it's still radically different. That difference is greater than the one between product and project, commercial and noncommercial, closed and open. It's a difference in nature itself.

In the past, I've characterized Linux's nature as that of a building material. Furthermore, I've characterized it as natural building material, similar to wood or granite. I've said it occurs in human nature, produced by human thought and work. I still believe all that is true. But now I believe it goes deeper than that.

This is what I learned from Andrew and Ted. After sitting in on their sessions (which ran many hours--the cruise was something of a crash course at sea), I realized that Linux's nature, as a building material, is akin to that of a species.

As I write this, the creators of commercial OSes are laboring to anticipate forms of work that nobody is doing yet. Even OS X, which sits on a base of BSD, is being altered constantly to anticipate purposes nobody is pursuing yet. It's the same with Windows developers working on Vista, the Windows successor due for release next year. Theirs are products that practice Marketing 101 for consumer electronics: create something that mothers a need. For Apple it might be HD video editing. For Microsoft it might be multilayered Web services. For Linux it might be...

How about..."Stamping out bugs"?

That was Andrew Morton's answer to a "What gets you excited?" question on the boat. His answer was a huge clue for me, because I realized Andrew's job is almost entirely reactive rather than proactive.

I used to wonder why Linus, when he spoke on earlier cruises, would answer so many questions with "I don't care", and then explain why he didn't care by saying "That's user space". At the time I thought this simply was a way for Linus to confine the scope of his interests and keep distractions away. Which it was, of course. But it was also something much sharper than that, something that is becoming clearer to me now.

User space is where commercial products live. Right now, on the desktop and the laptop, most of the world's users are working on Windows or OS X. Andrew doesn't doubt that Linux will get there too--eventually. But he also isn't holding his breath waiting, because he knows evolution moves at an infuriatingly slow pace.

When I told him about my experiences with different distros on my IBM ThinkPad T40, and why I ended up going with the Novell (SUSE) desktop, he dismissed my personal example as a misleading one. Instead, he put the matter in the deep perspective of evolutionary time:

Kernel development is not about Moore's Law. It's about natural selection, which is reactive, not proactive. Every patch to the kernel is adaptive, responding to changes in the environment as well as to internal imperatives toward general improvements on what the species is and does.

We might look at each patch, each new kernel version, even the smallest incremental ones, as a generation slightly better equipped for the world than its predecessors. Look at each patch submission--or each demand from a vendor that the kernel adapt to suit its needs in some way--as input from the environment to which the kernel might adapt.

We might look at the growth of Linux as that of a successful species that does a good job of adapting, thanks to a reproductive cycle that shames fruit flies. Operating systems, like other digital life forms, reproduce exuberantly. One cp command or Ctrl-d, and you've got a copy, ready to go--often into an environment where the species might be improved some more, patch by patch. As the population of the species grows and more patches come in, the kernel adapts and improves.

These adaptations are reactive more often than proactive. This is even, or perhaps especially, true for changes that large companies want. Companies such as IBM and HP, for example, might like to see proactive changes made to the kernel to better support their commercial applications.

Several years ago, I had a conversation with a Microsoft executive who told me that Linux had become a project of large commercial vendors, because so many kernel maintainers and contributors were employed by those vendors. Yet Andrew went out of his way to make clear, without irony, that the symbiosis between large vendors and the Linux kernel puts no commercial pressure on the kernel whatsoever. Each symbiote has its own responsibilities. To illustrate, he gave the case of one large company application:

He added, "On the kernel team we are concerned about the long-term viability and integrity of the code base. We're reluctant to put stuff in for specific reasons where a commercial company might do that." He says there is an "organic process" involved in vendor participation in the kernel.

Earlier this year, I had a conversation with IBM's Dan Frye in which he said the same thing. He also said that it had taken IBM a number of years to learn how to adapt to the kernel development process rather than vice versa. Andrew explains:

But it has to be good for the kernel. And good for supporting, as Andrew puts it, "serious applications".

Kernel space is where the Linux species lives. User space is where Linux gets put to use, along with a lot of other natural building materials. The division between kernel space and user space is similar to the division between natural materials and stuff humans make out of those materials.

More clues came from the language being used. Tree. Fork. Branch. Bug. Viability. Longevity. Even kernel is a term from nature.

Years ago I got to talking with Jackson Shaw, then of Microsoft, about embedded operating systems. When he asked me to name the three top applications for embedded Linux, I countered with, "Name the three top applications for lumber." I enlarged my point by adding, "Linux grows on trees." Rather than argue with that, Jackson ran with it, coming to the conclusion, "Linux doesn't grow on trees. It IS trees."

After hanging out with these wise men for a week on a boat, I'm starting to understand a bit more about how those trees evolve.

IT Conversations - Doc Searls: DIY-IT

"Cruising the Kernel with Andrew, Ted and the Gang, Part I"

"Cruising Without a Bruising"