What is the value proposition of an embedded operating system? That it is real-time? That it offers high-performance? That it is scalable? That it is reliable? That it is configurable? That it offers broad tool support? That it offers fast time-to-market? That it is well understood? That it is economical? Certainly these attributes are part of what is a longer list of features. After more than three years of bustling activity among software vendors and OEMs to use Linux as an OS for embedded devices, it is time to ask some hard questions. A recent report from Embedded Forecasters has raised some issues regarding embedded Linux that need to be addressed in a straightforward manner.

The crux of the report, which is titled “Total Cost of Development” is that it proposes a model by which to judge the cost of development for embedded development platforms. As examples, it compares the cost of development between OEMs who used Windows CE.NET or Windows XP Embedded and those who used various distributions of Linux. Using a simple formula of man months spent on a given project multiplied by the monthly cost of a developer on that project led to the startling results that not only did the average Linux project take over six months longer to complete; it also cost a staggering million dollars or more to finish. While this same model can (and should) be applied to other embedded development platforms, the implications for embedded Linux are dismaying. The report can be downloaded for free from www.embedded-forecast.com.

One of the biggest attractions of Linux has been its royalty-free availability and the availability of source code. Yet the study points out that to make up the development cost difference by savings on royalties would require selling over 300,000 copies. In addition, there can be other software components that may require licensing, which cuts into the royalty-free advantage. Perhaps the most significant aspect of the analysis — which is not its central focus — is the issue of tools and support. Pricing for Linux tools is uniformly higher than that for Windows tools, as is the cost of support, which can be either per-incident or by subscription, depending on the vendor. In addition, it is no secret that tool chains for embedded Linux are simply not nearly as mature, as integrated or as feature-rich as those for other commercial RTOSs.

One other disturbing result of this study is that some of the OEMs using Linux for embedded projects appeared not to understand what is required for a successful embedded project. Of those using commercial distributions, there was a mixture of OEMs selecting desktop distributions and others using distributions intended for embedded development. About a third actually went to the Web to “roll your own” downloads to start their projects. However, time to market and number of engineers on a project was not impacted significantly enough in any of these cases to compete with the Windows numbers.

It defies common sense to imagine that people in a highly competitive market with time windows as narrow as they are in this industry would start an embedded project by assembling an operating system with components downloaded from the Web. It makes far better sense to go to an embedded Linux vendor — or at least is evidence of the right instincts — but even these results were disappointing. It is quite possible that vendors of embedded Linux platforms will eventually offer mature, integrated tool chains and economical support options. As a number of embedded Linux vendors have made and are making good-faith efforts in that direction and improvement is rapid, that could very well happen, but time is not on their side. While developers may see some very valid reasons for choosing Linux, it is certainly better to start with a platform that has been tailored to the needs of embedded developers by vendors who understand those needs.

And if they should succeed, what then is the value proposition for Linux over the alternatives? There are already very good tool chains that can be used for developing embedded Linux systems. However, their vendors would rather support their own RTOS offerings and not get drawn into being Linux experts. And there are also a number of vendors whose RTOSs are royalty-free and who make the bulk of their revenue on tools and support.

I strongly suspect that the real culprit is the “siren song of open source,” which has lured many to their doom. Open source is the fool's gold of the software world. The assumption of those drawn to open source is that there is all this software that has been created for free by all these devoted people and somehow we are now going to use this code to make ourselves rich. So we pour millions and millions into “productizing” it and find that the code itself is still free and we must offer some other value proposition such as tool offerings and excellent support.

But if I, the developer, can buy one copy of your free source code and spend a one-time amount on a feature-rich tool chain, why should I pay any more to the Linux vendor? Ah, yes — support. That's a quandary. I can't get support from a tool chain vendor who doesn't support Linux and for support I have to pay a Linux vendor who doesn't have a mature tool chain. But at least I have the kernel source. That's my teddy bear. Of course if I change the kernel source, even the Linux vendor can't help me. But it's royalty free. However, if I eat up all my royalty savings in the increased cost of development, is there any way out?

No.

Not unless you're a Sony, an IBM, a Philips or a large organization that has the resources to hone and support Linux independently and in-house. Companies of this size do have a big chance to make Linux work profitably for them. Also, it looks like carrier grade Linux has a pretty bright future — in the context of large server-based systems. When this new CELF consumer electronics consortium does eventually produce a common distribution, it may accrue to the benefit of smaller OEMs, but that is some time off. The difference is that these companies will be burying Linux inside their products, adding value and amortizing their costs over really large volumes in the consumer market and that may be the most profitable way to go for them. That is, however, quantitatively different from the situation facing most embedded OEMs.

The purpose of these remarks — provocative though they may be — is not to trash Linux, but to challenge some assumptions that have not yet been challenged and to point out that there are many factors affecting the choice of a development platform. Some of these are emotional (if not to say “religious”) issues. But if you're betting your company's future, you'd better be sure that the choice, be it Linux or some alternative, is grounded in reality and sound economic and technical considerations.

In that regard, I have one major concern with the study and that is the source and nature of the underlying data. This is likely to bring up all sorts of issues, especially criticisms of what was not considered (e.g., size and complexity of various projects). Therefore it is important to know how the data was gathered and by whom as well as to know the scope of the data to allay suspicion of selectiveness (conscious or not). Unfortunately, although I am aware of the source of the data, it was disclosed to me off the record and I cannot reveal it. However, I call here and now for all parties to put their cards on the table. Failure to do so will only create more distrust and uncertainty and lead to unfounded accusations that will not further the open discussion of the real issues that concern this industry.

This column, which originally appeared in the August 2003 edition of RTC Magazine, is copyright © 2003, The RTC Group. Reproduced by LinuxDevices.com with permission of The RTC Group.

---

About the author: Tom Williams is Editor-in-chief RTC Magazine. Having most recently held the position of Editor-in-Chief for Embedded Systems Development, a Penton publication, Mr. Williams joined RTC with over 20 years of industry knowledge. He has worked for successful publications such as InfoWorld, Dr. Dobbs, ED, CD and ESD. He earned his master's degree from Washington University and completed his post-graduate work at Bonn University.

---

SPECIAL EDITORIAL SET:
Analyzing the EMF “Total Cost of Development” report

---

These three editorials originally appeared in the Aug. 2003 issue of RTC Magazine:

• Concerns about embedded Linux
  by Tom Williams, Editor-in-Chief, RTC Magazine
• Another perspective to the report on cost of development
  by Rick Lehrbaum, Editor-in-Chief, LinuxDevices.com and WindowsForDevices.com
• Embedded Linux and the end of amateur hour
  by Victor Yodaiken, founder and CEO of Finite State Machine Labs

---

Add your voice to this discussion!

---

Do you have comments or questions on this story, or on the EMF report?

talkback here
But please, before you jump into this discussion, read the full report! You can download it from the EMF website (requires free registration).

---

 
This article was originally published on LinuxDevices.com and has been donated to the open source community by QuinStreet Inc. Please visit LinuxToday.com for up-to-date news and articles about Linux and open source.