Fire, Brimstone and Real-Time Linux

Debate continues over the best approach to real-time capabilities and the Linux kernel.

Though fewer than 10% (or less) of embedded Linux applications actually require real-time enhancements or add-ons, articles and discussions on that subject invariably spark passionate debate. For whatever reason, the topic is a magnet for what might be characterized best as a sort of religious fervor. So it is with some tiptoeing through a minefield that this month's embedded column begins with several topics related to real-time Linux. Hold on to your hats.

VDC Sees Real-Time Linux Support Opportunity

In their recently published research report “Linux's Future in the Embedded Systems Market”, Venture Development Corp. (VDC) concluded that the availability of real-time solutions for Linux are needed to accelerate the broad adoption of Linux in embedded designs.

The report analyzes the current size and future growth of the worldwide market for embedded Linux software solutions ( According to the study, the dominant factors favoring the use of Linux in embedded projects include the availability of source code, royalty-free options and reliability. On the other hand, VDC found “real-time limitations” to be the most common issue cited by embedded developers as inhibiting their adoption of Linux for future projects.

Here is a list, in ranked order, of what developers told VDC were their main concerns with respect to using Linux in embedded systems and devices:

  1. Real-time limitations

  2. Doubts about availability and quality of support

  3. Fragmentation concerns

  4. Doubts about vendor longevity

  5. Footprint size

  6. Other

The Great Real-Time Linux Debate (Redux)

The usual real-time debate erupted shortly after Embedded Linux Journal published the third article in a series by Kevin Dankwardt on real-time Linux technologies. Here's an outline of the sequence of reactions and responses from key players in the real-time Linux market, followed by a pointer to where you can read them all on-line:

  • MontaVista Software's Kevin Morgan issued a response to Dankwardt's article in which he offered “a few clarifications (or points of view)”.

  • Victor Yodaiken and Matt Sherer (of FSMLabs) reacted to Kevin Morgan's response to Dankwardt's article, taking exception to Morgan's assertion that RTLinux is “not appropriate for the placement of comprehensive applications”.

  • Kevin Morgan clarified the status of MontaVista's kernel preemption enhancements and responded to several other issues raised in Yodaiken and Sherer's earlier comments.

  • Karim Yaghmour provided “the RTAI perspective”--drawing attention to the nature of the API, the usability of the methods and distinctions in the overall openness of the specific approaches being compared.

  • Doug Locke, TimeSys' VP of technology, contrasted his company's preemptible Linux implementation with the one pioneered by MontaVista and commented on several aspects of the preceding debate.

You can access all the above, including the original three-part ELJ article by Kevin Dankwardt, from this summary page:

Still More on Real Time

Clark Williams of Red Hat wrote a whitepaper titled “Linux Scheduler Latency”, in which he compares the performance of two popular methods for improving Linux kernel preemption latency—the preemption patch pioneered by MontaVista and the low-latency patch pioneered by Ingo Molnar—and discovers that the best approach might be a combination of the two (

The ADEOS Project announced its first release of ADEOS, a hardware abstraction layer that allows a real-time kernel and a general-purpose kernel to coexist on one CPU. The announcement claims that “RTAI will eventually use ADEOS services, thus offering a real-time kernel based on a principle clearly different from the 5,995,745 US Patent”, aka the “RTLinux patent” (

Victor Yodaiken published a whitepaper that points out the disadvantages of dealing with the issue of “priority inversion” in real-time systems by means of a commonly used method known as “priority inheritance”. Priority inversion refers to the situation when a scheduled task must wait for a lower-priority task to complete. In the whitepaper, Yodaiken describes “the classical nightmare case” of priority inversion as being “when a low priority task owns a resource, a high priority task is blocked [and] waiting for the resource, and intermediate priority tasks keep preempting the low priority task so it cannot make progress toward releasing the resource.” Yodaiken says the technique of priority inheritance is intended to allow:

a task that is blocked waiting for a resource [to pass] its priority down to the owner. The low priority task is [thus] considered to be acting on behalf of the highest priority blocked task and inheritance prevents intermediate priority tasks from interfering.

However, “priority inheritance is neither efficient nor reliable”, the paper argues, and its “implementations are either incomplete (and unreliable) or surprisingly complex and intrusive”, asserts Yodaiken (