SANTA CLARA, CALIF. — Analysis — MontaVista's inaugural partner event this week attracted 400, including many long-time embedded industry vets. Well-run and successful, the “Vision” event nevertheless underscored the modest role embedded developers play in Linux development, despite an explosion in… consumer spending on Linux-based devices.

In technology, consumer spending recently outstripped corporate spending, noted Freescale VP Dan Cronin in the event's first partner keynote. Yet, the “great bulk of kernel development is done by server companies,” noted Linux 2.6 kernel maintainer Andrew Morton in the second keynote.

Together, these two notable statements dovetailed in an interesting way with MontaVista's primary marketing message, delivered in the opening keynote by CEO Tom Kelly. Kelly said kernel.org Linux is not ready for production use in products, contrasting MontaVista Linux as being real-time enough for the Space Shuttle, and routinely used to program and collect data from in-patient medical devices.

Morton hardly disagreed. While not in any way endorsing MontaVista Linux in particular, or even commercial Linux distributors in general, he did suggest that pure kernel.org Linux best behooves only those products likely to have their OSes upgraded by the user — desktops and servers, and a subset of embedded products used as components in devices — chips, boards, and ATCA chassis, for example.

Morton (right) chats with Crucial Security engineer Wyatt Banks (left)
(Click to enlarge)
Morton's talk was the highlight of a two-plus day event that featured a wealth of technical sessions and tutorials, along with a vendor expo staffed by many highly knowledgeable people, some with decades of embedded marketing experience dating back to Microtech and even Ready Systems. Along with 400 attendees, the event drew about two dozen MontaVista partners, including Freescale, TI, Aonix, Virtutech, Enea, various electronic component distributors, VirtualLogix, Trango, Open Kernel Labs, Sun, Trinity Convergence, and others.

Much high geekery was on display, including this Nokia N800 running Qtopia Phone Edition (and yes, that's a Trolltech Greenphone in the background)
(Click to enlarge)

In his big talk, Morton said that only a few developers are actively working to improve Linux's embedded characteristics, while the rest are just doing “damage control,” trying to make sure that new server and desktop features are as non-injurious as possible to embedded architectures and performance expectations. And, Morton admitted that the 2.6 kernel is “getting fatter.” He said, “We're aware of the problem.”

The rest of his talking points mainly seemed aimed at inspiring more embedded participation in mainline Linux kernel development. Lumping consumer in with embedded, Morton said, “Embedded is under-represented in kernel development, while there's an over-representation of servers.”

Some of Morton's points, messages, requests, and tips included:

• Just seeing if your devices will boot new kernel releases (and feeding back, if not) is the most important contribution embedded developers can make. “It's better to run one test on 1,000 machines, than 1,000 tests on one machine,” he said.
• Users posting bug reports to kernel lists should ignore responses like, “Why don't you write a patch?” Morton said, “There's no coupling between discovering and fixing bugs.”
• Older kernels were already pretty good embedded OSes, helping explain why more hasn't been done to improve Linux for use in devices lately. However, except for “single-product” companies, freezing on an old version of Linux is not a good long-term strategy, in part because kernel developers are highly motivated by self-interest to support current releases.
• Morton expects all of Ingo Molnar's real-time patches to make it into the mainline kernel, but expressed frustration that it's happening slower than expected.
• The legacy of “patch hoarding” — maintaining embedded patches outside of the mainline kernel — should end. People should send their patches to the LKML (Linux kernel mailing list), where etiquette has reportedly improved. Morton also iterated and re-iterated the invitation to send patches privately to him for a candid appraisal.
• Morton listed many benefits to “going mainline,” but failed to note the likelihood of positive press being generated for companies contributing kernel enhancements.
• Companies hoping to merge patches should designate one or two developers to interface with mainline via the LKML (Linux kernel mailing list).
• As many embedded developers as possible should read LWN and the LKML, and provide feedback to kernel developers. “Don't just lurk. Don't just suffer in silence,” he said.

While encouraging more embedded involvement in kernel development, Morton admitted that he didn't really expect much to change. Pointing to low margins in the device market, he said, “I don't expect the underlying economics to change.”

It was difficult not to consider the example of MontaVista, one of the first companies focused solely on embedded Linux. During Linux's meteoric rise to the top of the embedded OS heap, MontaVista has remained a fairly small company, while other commercial embedded Linux distributors have also struggled.

It's also tempting to wonder whether greater economic success for commercial embedded distributors like MontaVista might have resulted in more parity between server- and embedded-oriented kernel development. MontaVista currently employs about 10 developers who regularly contribute to mainline. One of its competitors recently lost a considerable amount of engineering talent, along with a long-time CEO and investor. Meanwhile, server companies like HP and IBM each employ hundreds if not thousands of Linux developers.

On the other hand, it's always possible that real success and growth could be just around the corner for MontaVista and other embedded Linux providers. Sources close to MontaVista point to two outstanding growth quarters following founder Jim Ready's decision to make room at the top for veteran turn-around artist Tom Kelly and his sidekick Rusty Harris, EVP of sales. “People have criticized MontaVista's business model through the years, but we really don't know whether it's good because they've never executed on it before,” one anonymous source close to the company said.

Another positive is the timely announcement of Mobilinux 5.0 right before MontaVista's Vision event. In his talk, Morton praised MontaVista for its decision to use a “fairly contemporary” kernel; perhaps a back-handed reference to MontaVista's sticking with 2.4 for so long. In the past, MontaVista has used the same kernel version across all its vertical-market Linux flavors, while Mobilinux 5.0 broke ranks with 2.6.18-based “Pro” (MontaVista Linux Professional Edition) in jumping to a 2.6.21 kernel.

Still, MontaVista gets no runtime royalties for its Linux kernel — only developer seat fees, leading some to question the value of its large investments in the mobile space, where it claims to have been shipped in 45 million devices. Yet another problem is that MontaVista's narrow focus on the kernel is feared to be below the “value line” of what some mobile phone companies — LiMo Foundation members, for example — someday expect to get for close-to-free out of Linux.

To date, MontaVista has received over $90 million in investments, but has yet to report that it has turned an annual profit. Kelly admitted that the company is not yet ready to be acquired or go public.

Kelly and Ready, at the Vision event
(Click to enlarge)
Still, Kelly insisted that “business is up, in both revenue and demand.” He also noted that a lot depends on fourth quarter results. “If we can string together a strong Q4 with our historically strong Q1, we'll be in a very good position,” he said.

Jim Ready, company founder and CTO, added, “Kernel.org is not in the business of delivering products. They're in the 'technology-go-fast' business. More and more companies are realizing that.”

Next up for MontaVista is an ambitious roadshow, more evidence the company is working harder than ever to establish a solid footing for itself in the sometimes rocky and marginal embedded software and tools market.

— Henry Kingman

---

 
This article was originally published on LinuxDevices.com and has been donated to the open source community by QuinStreet Inc. Please visit LinuxToday.com for up-to-date news and articles about Linux and open source.