Foreword: This article was contributed by an embedded product development consulting firm that often works with Linux. It outlines the role that chip vendor-supplied evaluation kits play in determining time-to-evaluation and ultimately time-to-market, with a candid appraisal of the kits available from three specific chip vendors.

---

The role of evaluation kits in embedded product time-to-market
by Alan R. Weiss

Our company, Synchromesh Computing, has substantial experience with SDKs and evaluation kits. We built our business, originally, on benchmarking and performance analysis, and moved into also offering software development and consulting. We have used hundreds of SDKs and reference platforms. Because of this, we have developed an understanding of what constitutes a complete and viable offering:

• Development tools that are “production-grade” and that produce good quality and fully usable code
• Evaluation kits (working boards, gerber files, documentation) that come complete and are easy to set-up
• Reference designs, which are complete solutions (including production-ready software), that can be taken to market in a very short time by Original Design Manufacturers (ODMs), or integrated/replicated easily into a design by a system provider
• Market-targeted software platforms that include components (hardware and software) aimed at specific applications
• A robust third-party ecosystem for operating system software, software development tools, middleware, and technical support
• Software development services to provide consulting and design assistance

Figure 1: Elements of a complete solution
(Click to enlarge)

All of the above must be evaluated for functionality, performance, power consumption characteristics, accuracy, interoperability, flexibility, ease of licensing — and most importantly, quality. In addition, they all must be optimized for the targeted platform. They must be in place, delivered on time, and cost effective to allow designers to get complex products to market quickly.

More and more designers today are selecting the components they purchase based on the amount and quality of available alternative software solutions, customer care, and support that will be given post-market. According to an EE Times market research report published in September of 2007, the ecosystem surrounding a processor has been number one in importance for three straight years, hovering between 58 percent and 65 percent. The chip itself is only number two, coming in at between 34 percent and 42 percent over the past three years.

Figure 2: What's more important, in choosing a microprocessor?
(Data source: EETimes, Sept. 2007 issue)

Sure, processor companies still sell processor features and performance (and power consumption), but time to market considerations, solved in large part by the ecosystem and by offering high quality kits, are becoming an increasingly important factor in such purchase decisions.

Third party support isn't just a lining up a bunch of logos on a website, but rather making sure that third party software and hardware work well on the target board, and work well together. Freescale's Market Ready Validation Program for i.MX processors and AMCC's Acceptance Testing Program both assure that third party software works. The AMCC website also offers software options for AMCC processors broken down by processor, by type of service or software needed, and in other ways that make searching easier.

As for the quality of evaluation kits, our company has seen a full range, in the course of ten years in business. We've seen a surprisingly large number of poor quality SDKs and reference platforms from reputable companies. We've seen:

• Power consumption at idle that is higher than that when running an application.
• Kernels and compilers that are woefully out of date
• Documentation that confuses rather than illuminates
• Technical support that borders on the abusive

Since we often bid an evaluation or benchmarking contract based upon a fixed price, every hour — or week — spent fiddling around and trying to get a board and software tool set to work properly is lost revenue for us. And for product designers, these types of delays can easily mean a lost market window or even bankruptcy.

Designers need code that they can actually use and re-use, not just “sample code.” According to one senior software engineer we spoke with, “We don't just want throwaway reference code for U-Boot and Linux. We want production quality code from a commercial supplier, who is guaranteed to maintain that going forward with new kernel revisions and new feature editions. We need to be able to use that code with confidence in our security and storage products.”

The engineer added, “When we want technical support, we want to go to one entity and they take care of it. We want this included. Don't make me hunt around for it on some third party website.”

Most tech companies today do offer some sort of technical support, even if it is outsourced to the other side of the globe. Few seem to take the time to really set up a system to handle real world customer problems, though, and even fewer actually seem to rigorously test a product's functionality from opening the box all the way through the set up process. Almost none appear to actually test their customer service as if they were a customer themselves.

How chip providers rate

One company we work with frequently stands in front of the pack in their efforts to make their out-of-the-box experience as pleasant as possible. Applied Micro Circuits Corporation, otherwise known as AMCC, is a provider of PowerPC embedded processors. AMCC Evaluation kit products can consistently be up and running in anywhere from about ten to about 30 minutes from the moment the boxes arrived. This is more than remarkable in our experience, and especially when compared to AMCC's competitors.

Figure 3: AMCC kits include an easy to use browser interface
(Click to enlarge)

AMCC has also proven to be strong on ecosystem support, with a well-organized website listing partner support resources in a single place. They provide benchmarks with their kits, and even a browser interface (pictured above) to organize documentation. They appear to know what their buyers need. If all else fails, they have an extensive support system.

At Freescale Semiconductor, different divisions have different approaches to what that company calls “reference platforms.” Until recently, the popular ADS board was offered across a range of processors, including PowerPC and i.MX. The i.MX board, an ARM-based system, suffered from a number of software problems, including poor operating system support and old compilers. In addition, the large form factor boards were showcasing processors used in mobile handset applications, and customers couldn't really “see” how the i.MX31 (for example) processor could be packaged into a mobile phone. This was no doubt a legacy from the days when ADS platforms were used to demonstrate PowerPC processors. Freescale responded just recently with their new Product Development Kit (PDK), which is a small form factor modular platform architecture that has vastly improved software and hardware components.

Figure 3: The new Freescale i.MX PDK

At Texas Instruments, the OMAP 2420 and 2430 development platforms come in very large boards and are only available through a single distributor. This distributor, GSCS, is a good company to work with, but is often kept in the dark about when development platforms are available. And, being a distributor, they know little about what software ships with the system. Often schematics are not included, making it difficult to measure power consumption (for example). Texas Instruments makes fine processors, and their Code Composer Studio is legendary amongst software developers for completeness, but making some products difficult to obtain, and not well supported, makes it difficult to recommend OMAP as a solution.

Conclusion

At Synchromesh Computing, we have concluded that speed-of-evaluation is a very important driver for helping end-product developers reduce time-to-market. Since time-to-market is the ultimate key to success in this business, those companies that put real time and effort into making their components' out-of-box experience for developers as rapid, efficient, and painless as possible will be those companies best-positioned for success in the 21st Century.

---

About the Author — Alan R. Weiss, has been in the computing and semiconductor and processor industry for 26 years. He has worked as a software engineer, manager, project manager, program manager, and software architect on operating systems, compilers, database systems, and embedded real-time software. He Founded EEMBC Certification Laboratory (ECL, LLC) in 1998, and founded Synchromesh Computing, LLC in 2003. Located in Austin, Texas, Synchromesh Computing provides software development, product development, benchmarking, testing, and certification services.

---

 
This article was originally published on LinuxDevices.com and has been donated to the open source community by QuinStreet Inc. Please visit LinuxToday.com for up-to-date news and articles about Linux and open source.