Longtime embedded industry leader Wind River is preparing to ship its first products aimed at post-development device management. The company's Field Diagnostics and Workbench Diagnostics products allow device vendors to remotely monitor, diagnose, and repair deployed devices, it says. They target manufacturing equipment makers and network infrastructure equipment vendors, initially.

According to Naren Nachiappan, VP of device management at Wind River, the idea behind the diagnostics product line is to eliminate the problem of having to reproduce software bugs in the lab. He explains, “People usually have to rebuild a brand-new target image with instrumentation, reload the target with the debug image, and try to capture data there. With the diagnostics capability we're announcing today, you can dynamically inject new diagnostic knowledge into running applications, and bring back enough information, including core dumps, so you don't have to reproduce the problem in the lab.”

Wind River's Diagnostics tool usage model
(Click to enlarge)
The Field Diagnostics tool was written as a J2EE (Java 2 Enterprise Edition) application that supports standard enterprise-class application servers, such as JBoss and WebLogic. It communicates over TCP/IP with “sensor point” agents that reside on deployed devices. These agents are available for VxWorks and as a loadable Linux module, and consume between 50KB and 100KB of memory.

The sensor points can be used for “dynamic instrumentation” in running applications, or to otherwise install hot-patches, down to the level of replacing individual functions. This functionally introduces minimal processing overhead, according to Nachiappan. “If you do not have a sensorpoint installed, there's no impact on the application. If you do, a single branch instruction is the overhead. Depending on how much state you want to save, that could vary from a few clock cycles to a few dozen. Binary sensor points have the lowest overhead, while a managed sensor point that saves a lot of state costs a little more.”

Once the problem has been reproduced in the field, and the relevant diagnostics collected, an available Workbench Diagnostics tool can help developers fix the problem. Nachiappan explains, “You can import a core dump into Workbench, and do a stack trace, and Workbench will map it back to the original source code.”

Once the offending code has been fixed, the Field Diagnostics tools can then be used to dynamically patch the running application, until such time as revised firmware images can be scheduled for installation.

Nachiappan says the diagnostics tools' architecture allows for the use of standard Linux security, including network encryption and secure authentication. Only a narrowband TCP/IP connection is needed, he says. Additionally, the local manager controls what information is sent back to the device manufacturer. Device makers could not exploit the tools to apply patches without customer knowledge, he adds.

Nachiappan says Wind River will market the tools initially to its largest customer bases, network infrastructure equipment providers, as well as robotic manufacturing equipment providers such as Honeywell. He adds, though, that the product is “horizontal in terms of applicability. It's not tied to any particular vertical.”

The Diagnostics tools will also be marketed initially to large QA (quality assurance) and testing departments, because of a claimed ability to shorten the edit, compile, debug cycle. Nachiappan elaborates, “It's a great way for QA and test engineers to inject faults. It can modify the behavior of systems from the inside. This is an interesting way for test engineers to do whitebox testing and get high levels of code coverage.”

Ericcson is one early beta tester using the tools for device testing and development, according to Nachiappan.

A 2003 MontaVista whitepaper on runtime patching of Linux applications for high-availability systems can be found here.

Availability

The Diagnostics tools are beta-testing now at several customer sites, according to Wind River.

---

 
This article was originally published on LinuxDevices.com and has been donated to the open source community by QuinStreet Inc. Please visit LinuxToday.com for up-to-date news and articles about Linux and open source.