Another perspective to the report on cost of development
Sep 10, 2003 — by LinuxDevices Staff — from the LinuxDevices Archive — viewsDespite the title of the report, EMF's “Total Cost of Development” study is really about using the author's TCD metric to compare the development efficiency of Windows Embedded and embedded Linux.
My intent is not to diminish the excellent tools and software that Microsoft has developed for embedded systems and devices. However, in reviewing the EMF report I came to the conclusion that it is incomplete, inaccurate, and unbalanced, and that its conclusions do an injustice to the embedded Linux market. Here are a few of my observations in reviewing the report; please visit my more extensive online article, noted below.
1. Qualitative comparison of development environments and tools
The report points out the availability of Microsoft's Windows Visual Studio .NET application development environment, which it compares to the high quality IDEs and tools developed by many software vendors in the embedded and real-time OS market. But Linux, the report says, has “few” such tools, and “there are even fewer that take into consideration the unique requirements of the embedded designer.” How does “few” equate to less than Microsoft's “one?”
The report further distorts the picture of respective development environments. Rather than including examples of some of the solid IDE/toolkit development environments from the likes of LynuxWorks, Metrowerks, MontaVista, and TimeSys, it confines its comparison to one between Microsoft's commercial IDE with the freely downloadable GNU tools. I also suspect that a table — glaringly missing in the report — comparing BSPs between Windows Embedded and embedded Linux would be heavily weighted in favor of the latter.
2. Cost comparison of development environments and tools
The average cost comparison tabulated in Appendix C offers an interesting sampling of “vendor-supplied or published pricing” for a few of the many embedded Linux toolkits alongside Microsoft's Windows Embedded toolkits. One wonders how up-to-date and complete the information is, after noting the inclusion of Lineo, the absence of Metrowerks, the absence of TimeSys and the absence of a host of others. Besides, how many embedded vendors — especially ones offering royalty-free products — are likely to come right out and tell you how low their prices will go when you begin negotiating multiple developer seats or pit them against competitors?
3. Cost comparison of OS royalties
The report dismisses — without explanation — the royalties associated with Windows XP Embedded. “For the purposes of runtime royalty comparison, only Windows CE .NET and embedded Linux will be considered” (Pg. 15, first paragraph). Suddenly, XP Embedded royalties are now gone from the report! By my reckoning, XP Embedded royalties are roughly 1000 to 3000 percent those of CE .NET — yet somehow the EMF report feels that the royalties associated with half of its source data are somehow irrelevant.
The study goes on to burst the myth that embedded Linux software is royalty-free.Table 6 lists components of Windows CE, which supposedly would require royalties in a comparable embedded Linux system. Two questions:
(1) Why are key Linux components such as real-time, web browser, and others indicated as royalty-based when multiple free versions of these are readily available?
(2) Does it seem reasonable that the table would exclusively contain functions that are contained within Windows CE and not vice-versa for Linux?
4. TCD results based on a survey of 100 projects
In my opinion, report misrepresents the realities of embedded Linux as an OS, and of the commercial embedded Linux software market, by failing to address the unique character of the embedded Linux market and of embedded Linux itself. Unlike other embedded OSs and RTOSs, embedded Linux is not dominated by a single vendor. Choice is one of its greatest strengths. Companies can't be held captive to a single solution or vendor.
From my experience, I suspected most of the embedded Linux projects considered had been developed without using commercially supported toolkits — in contrast to the Windows Embedded ones which all used Microsoft's commercial toolkits.
I therefore asked the report's author how many of the Linux projects used freely downloaded software, and how many used commercially obtained toolkits. The answer: of the 45 projects surveyed, “18 OEMs purchased their version directly from a Linux vendor.” That comes to just 40 percent. This means the TCD data compared 50 projects using Microsoft's toolkits with a mixture of Linux approaches — 18 based on purchased commercial toolkits, and 27 based on free downloads.
Another factor is that on average the projects were probably done nearly two years ago, and possibly longer ago than that. In other words, even the projects done using commercially supplied/supported IDEs/toolkits were using relatively immature development environments and toolkits — since commercial embedded Linux has only existed for about three years. Thus, many of the Linux projects — especially the ones with the longest time-to-market values — did not benefit from today's quality embedded Linux toolkits.
In short, I believe the TCD results of the EMF report are due to a comparison of development projects conducted with — and without — commercially supplied/supported IDEs/toolkits, rather than due to the underlying OS technology. Interestingly, that's almost exactly what Krasner says in the last paragraph of the report's executive summary!
I urge you to read the entire EMF report, Tom Williams' editorial, and my expanded discussion of the report. And, add your own opinions in the talkback thread provided at the end of my online article.
This column, which originally appeared in the August 2003 edition of RTC Magazine, is copyright © 2003, The RTC Group. Reproduced by LinuxDevices.com with permission of The RTC Group.
About the author: Rick Lehrbaum created the LinuxDevices.com, WindowsForDevices.com, and DesktopLinux.com websites. Rick has worked in the field of embedded systems since 1979. He cofounded Ampro Computers, founded the PC/104 Consortium and was instrumental in creating and launching the Embedded Linux Consortium.
Analyzing the EMF “Total Cost of Development” report
These three editorials originally appeared in the Aug. 2003 issue of RTC Magazine:
- Concerns about embedded Linux
by Tom Williams, Editor-in-Chief, RTC Magazine - Another perspective to the report on cost of development
by Rick Lehrbaum, Editor-in-Chief, LinuxDevices.com and WindowsForDevices.com - Embedded Linux and the end of amateur hour
by Victor Yodaiken, founder and CEO of Finite State Machine Labs
Do you have comments or questions on this story, or on the EMF report?
But please, before you jump into this discussion, read the full report! You can download it from the EMF website (requires free registration).
This article was originally published on LinuxDevices.com and has been donated to the open source community by QuinStreet Inc. Please visit LinuxToday.com for up-to-date news and articles about Linux and open source.