Article: An approach to solving the “device driver crisis”
Nov 21, 2003 — by LinuxDevices Staff — from the LinuxDevices Archive — 15 viewsForeword: This whitepaper describes an approach to solving the “device driver crisis” that plagues both the embedded and non-embedded computer system markets. SciTech claims its System Neutral Access Protocol (“SNAP”) device driver architecture, the result of over a decade of research and development, represents a new paradigm for device driver development that is radically different from… traditional methods and that can drastically reduce the complexity of OS- and processor architecture-related device driver support.
SciTech says its SNAP driver architecture centers around a unique Hardware Abstraction Layer (HAL), combined with pre-compiled binaries for each specific processor family that run on any operating system (including Linux / embedded Linux). According to SciTech, SNAP takes the concept of sharing source code and radically extends it — to sharing fully tested and certified portable binary driver images. Additionally, SciTech's SNAP tools are released under a commercial/LGPL dual-license, with a free download of the LGPL version available, enabling developers to use the tools to create free, LGPL drivers.
SNAP currently supports both x86 and PowerPC processor architectures, with work under way to support ARM and MIPS. On the x86, SNAP currently supports over 200 different graphics chipsets on dozens of operating systems and APIs. On the PowerPC platform, SNAP is expected to support at least the top 100 graphics chipsets available to developers.
The whitepaper below presents an overview of SciTech's SNAP technology, followed by short descriptions the various SNAP products, including the Industrial Embedded Systems (IES) offering for embedded devices.
An Accelerated 32-bit Graphics BIOS
for The Industrial Embedded Systems Market
- The Traditional Driver Development Model
- The “Driver Crisis”
- Solving The Driver Crisis with SNAP
- SciTech SNAP, Certifying the Solution
- SciTech SNAP Products
- About the Authors
The Traditional Driver Development Model
A device driver is the software component that allows operating systems and application software to use a hardware device. Under ideal circumstances, every user would have a properly functioning device driver regardless of which company manufactured their hardware, its age, which operating system they choose, or what processor they are using.
Overview of the Traditional Device Driver Model
The traditional device driver development model consists of three layers, as depicted in Figure 1:
Fig. 1: Traditional Operating system Driver Model
Application programs communicate with the operating system via Application Programming Interfaces, or APIs, unique to each operating system. The operating system in turn translates these service requests for a specific device through a device driver. The device driver layer implements all device specific code as well as the high level features that the operating system requires for that class of hardware device. The device driver layer is defined differently for each operating system.
A major shortcoming of traditional device driver development is the lack of unified source code between drivers for different operating systems, and between drivers for diverse hardware devices. Differing device driver layers between operating systems usually means that during driver development, the source code for the equivalent drivers on each operating system is either developed from scratch or code for a similar operating system is ported to the new operating system. The end result is that the driver source code for each operating system is different, even though the drivers for different operating systems provide essentially the same functionality.
This means that any device driver bug fixes or performance enhancements discovered on one operating system must be re-discovered and fixed independently for each supported operating system or hardware device. As the number of devices and operating systems supported increases, it becomes increasingly unmanageable for a hardware vendor to maintain device drivers for all but the most popular operating systems.
Device drivers require complex testing and verification to ensure correct operation. Since this testing process is so complex and time consuming, hardware vendors often do not update older drivers with new features and fixes. In some cases, the same driver can be built to support multiple hardware devices, however due to the large test matrices involved, the driver is often not completely tested with older hardware during each new release cycle. The end result is that the quality of the device drivers for older hardware gets worse over time as the testing and verification process becomes more time consuming, or updated device drivers are not made available for older hardware devices.
The “Driver Crisis” is the dilemma the computer industry currently faces due to the lack of sufficient device drivers. Most computer users are affected by the Driver Crisis, since the functionality, stability and performance of their computer system is entirely dependent upon the availability and proper operation of the device driver for their specific hardware and operating system combination.
The Popularity of Open Architectures Precipitated the Driver Crisis
The current PC architecture, which evolved from the original IBM Personal Computer, is based on various industry standards. These standards allow multiple operating system vendors and hardware vendors to compete using the same basic hardware design. However the open flexibility and vast number of hardware devices available for the PC platform is one of the primary factors that has fostered the growth of the Driver Crisis.
Some computer system vendors such as Apple have attempted to avoid the Driver Crisis by building specialized, closed architecture computer systems running proprietary operating systems. Closed architecture computer systems have the potential to avoid the Driver Crisis since the manufacturer has control over all devices and the entire operating system. However even Apple is today feeling the pinch of the Driver Crisis as they try to support all the different PCI, AGP, USB and FireWire devices now available on the market.
Today, the following groups feel the effects of the Driver Crisis:
- The Operating system Vendor — The wide choice of available hardware generates a Driver Crisis for operating system vendors. The operating system vendors needs to ensure their operating system is compatible with hardware devices already in existence and those new devices entering the market.
- The Hardware Vendor — The need to support the various different operating systems and hardware platforms that their customers are using generates a Driver Crisis for the hardware manufacturer.
- The End User — The final result is that the drivers provided by the hardware vendors are often of less than adequate quality, or simply not available for their operating system or hardware platform. This in turn results in a Driver Crisis for the end user.
The Driver Crisis Worsens over Time
A device or operating system has different levels of support depending on its popularity and its life cycle stage. Over time, an operating system that may have once had good device support ages to the point where its device support is less than adequate. Conversely, a device driver will age so that new features and operating systems are not supported and the driver is no longer actively maintained.
These points are illustrated in Figure 2 below. For this diagram we consider a Tier 1 operating system to be a high volume platform such as Windows 2000/XP. A Tier 2 operating system is either a high volume operating system that has aged, such as Windows 95, or a new operating system that is not as popular, such as Linux or Mac OS X. A Tier 3 operating system is one that is very old, such Windows 3.1 and OS/2, or serves a vertical market, such as most industrial embedded operating systems like SMX and RTOS-32.
Fig. 2: Driver Aging over Time
Users purchasing new hardware running Tier 1 operating systems may start out with a good chance of having adequate driver support. However, over time, both the hardware devices and the operating system ages. As the hardware devices age, users upgrading to the latest operating systems find themselves moving into middle left group and the chance of getting good drivers becomes hit or miss. Likewise, as the operating system itself ages, it becomes a Tier 2 operating system and the chance of getting good drivers decreases. Eventually the operating system is relegated to Tier 3 and is eventually phased out or abandoned, and the chance of getting good drivers is pretty slim.
With the current frantic pace of hardware development, eventually most users end up in the bottom group within a surprisingly short period of time after they purchase their new computers. This leaves them with a slim chance of finding drivers that will work properly with their hardware.
Solving The Driver Crisis with SNAP
Due to the high costs involved with supporting multiple operating systems many hardware vendors choose to only support the most popular, high volume operating systems. This approach has led to the development of the traditional driver model – a fairly robust, single purpose driver designed to exploit the capabilities of a single operating system or a line of highly related operating systems. Conversely, the origins of SciTech SNAP can be traced back to the need to support various and highly divergent operating systems with comparable performance and feature sets.
To better understand the SciTech SNAP architecture, it is helpful to first see what it has in common with the traditional device driver architecture. Both traditional and SciTech SNAP device drivers have two main parts, an operating specific portion and a device specific portion. Using traditional device driver development techniques, the two driver components (operating system and device specific) are combined to form a single, physical device driver binary. At this point SciTech SNAP drivers make an important departure from the traditional driver architecture by physically separating the two parts into separate binaries as described below:
- The operating system specific portion
Every operating system defines a unique device driver interface for each class of hardware device it supports. Since each operating system defines the device driver interface for each class of device in a completely different way, this component is operating system dependent.
In SciTech SNAP this top-level component, called the 'Shell Driver', accepts native operating system requests. The operating system shell driver contains no hardware specific code and contains only the code necessary to link a specific operating system to the SciTech SNAP driver.
- The device specific portion
This is the device driver component that interfaces directly with a specific piece of hardware. Since the actual hardware device does not change when using different operating systems, this core code is essentially the same between operating systems and can be considered operating system independent.
In SciTech SNAP, this bottom-level component accepts generic requests from the shell driver and directly programs the specific hardware device. Since the SciTech SNAP driver contains no operating system specific code, it will work with any operating system shell driver. SciTech SNAP also goes one step further by making the device specific portions completely portable by using a binary portable module system, as described in the next section.
In traditional development, device drivers for different operating systems rarely share the same code and therefore fail to benefit from optimizations and bug fixes made to other related drivers. With SciTech SNAP much of the same code is shared across all supported operating systems. SciTech SNAP is able to do this by making the code binary compatible between operating system by building the hardware specific portion into a Binary Portable Driver.
A Binary Portable Driver is an operating system independent, Dynamic Link Library ('DLL'). The DLL concept is a standard feature on all modern operating systems, however the mechanisms used to create a DLL and the format of a DLL are specific to each operating system. Hence, a DLL built for Windows cannot be used under OS/2 or Linux, and DLL's created for those environments cannot be used under Windows. In contrast, the exact same SciTech SNAP Binary Portable Driver can be loaded and utilized on any operating system within a single processor family (and they can be recompiled for use on different processors).
One of the key benefits of this partitioned driver model is that the device specific components only need to be developed once for a particular hardware device, which drastically reduces the time required for development and certification of drivers for particular hardware device on multiple operating system.
Formalized Division Promotes Sharing
This modular design means that all optimizations or bug fixes implemented in the operating system shell driver are effective for all hardware devices, not just a single device. Likewise, any optimizations or bug fixes implemented in the SciTech SNAP driver are effective on all supported operating systems. Hence, this design allows the maximum amount of code sharing among different hardware devices and operating systems as shown in Figure 3.
Fig. 3: SciTech SNAP Driver Architecture
Not only does this model provide for increased productivity it has also proven to provide very high performance. Benchmarks show that SciTech SNAP drivers perform as well, or better in many cases than drivers written for a specific operating system and hardware device.
SciTech SNAP, Certifying the Solution
Certification of SciTech SNAP drivers is also done differently to the traditional methods. The design of the SciTech SNAP driver architecture divides the device drivers into two separate components, so likewise the certification process is also divided into separate steps. By separating out the hardware specific tests from the operating system specific tests, the test matrixes are dramatically reduced. This process cuts the total time it takes to certify the individual driver components, and dramatically shortens the time it takes to get a certified solution back out to the customer.
The complete certification process is described below.
- Certifying the binary portable hardware driver
To certify that the SciTech SNAP drivers are generating the correct output for a given set of input conditions, a sequence of complex stress tests are run on both the hardware device driver and a reference driver implemented entirely in software. The hardware and software generated output are then compared pixel by pixel to ensure that the hardware device driver is generating the correct output. This certification testing can be done on nearly any supported operating system, as the hardware specific drivers are completely binary portable. None of this testing is operating system specific. This process is illustrated in Figure 4.
Fig. 4: Certifying the SciTech SNAP Driver
- Certifying the operating system shell driver
Once the binary portable SciTech SNAP driver is certified, the next step is to certify that the operating system shell drivers work correctly when running on top of a certified SciTech SNAP driver. This is accomplished by running the operating system specific conformance tests (such as the Windows Hardware Quality Labs tests) on a select number of hardware devices. The hardware device chosen for this test are hand chosen by SciTech QA personnel and are representative of the different types of hardware devices supported.
- Certifying hardware for a specific operating system
Once all the major testing has been done as above, one more final step is done to ensure each hardware device is working correctly for a specific operating system. This is done with a simple boot and visual inspection test with both the operating system and the hardware specific components working together. Once our QA team verifies that the driver is working correctly under a specific operating system, that hardware device is stamped as certified and will be deployed in the next official release for that operating system.
Since all the rigorous testing on the hardware specific and operating system specific components is already complete, errors at this stage are rare. These tests primarily are intended to catch problems that have fallen through cracks during the battery of certification tests performed on the individual components. Errors discovered at this stage of testing indicate problems with the device specific testing procedures, and are treated very seriously. Both engineering and QA get involved to diagnose the problem and construct new tests to catch the error correctly in the tests for the device specific binary portable components.
SciTech's products have been in wide commercial use since 1993 and have been utilized by many of the worlds most innovative and pioneering companies, companies including Id Software, Valve Software, Virgin Interactive, Electronic Arts, Xerox, IBM, Intel, Compaq, ATI and many more. Today SciTech proudly calls many of these early adopters not only customers, but also partners. Along with these long-term customer/partners SciTech is also pleased to have the opportunity to work with many new and exciting companies, such as Trolltech, MAI Logic, On-Time Software and others to further develop the SciTech SNAP Graphics architecture and extend its use into new and exciting markets. The SciTech SNAP Technology is perhaps best known in the financial services, air traffic control and large enterprise server markets, where IBM licensed a special version of SciTech SNAP Graphics for OS/2 on behalf of its customer base worldwide. This license provides OS/2 users the ability to make use of a broad range of graphics devices through the end of life of OS/2.
- SciTech SNAP Graphics Enterprise Edition
Available for Linux (X11) and OS/2 systems, this version is specifically designed to ease the administrative overheads and support burdens associated with maintaining large enterprise computer systems. The ability to deploy a single graphics device driver system across the entire enterprise, regardless of installed hardware, is the biggest feature of this version.
- SciTech SNAP Graphics IES Edition
Available for Linux, DOS, OS/2, SMX, On Time RTOS-32, and other embedded operating systems, this version is a highly scalable solution for use in industrial and embedded systems development. SciTech SNAP Graphics IES can support application development using the X Window System (X11), or for more memory constrained environments using Qt/Embedded, SciTech MGL or direct to the hardware via the SciTech SNAP SDK. This version also allows the SNAP drivers to be configured to support a single chipset with a specific set of features or multiple chipsets with a wide range of available features depending on storage requirements.
- SciTech SNAP SDK
Available for all supported platforms, the SciTech SNAP Software Development Kit contains everything necessary for developing applications and operating system shell drivers using SNAP. You can develop applications that use the SciTech MGL library or talk directly to the hardware via the SciTech SNAP drivers using the SDK. The SciTech SNAP SDK is dual licensed under both a commercial license and the GNU Lesser GPL license.
- SciTech SNAP DDK
The SciTech SNAP Device Driver Kit contains everything necessary for developing and debugging custom SciTech SNAP device drivers. The SciTech SNAP DDK is intended for hardware vendors or developers who wish to build SNAP drivers for custom hardware themselves, rather than having SciTech develop the drivers. The SciTech SNAP DDK is dual licensed under both a commercial license and the GNU GPL license.
The SciTech SNAP SDK is currently available for download from SciTech's website, and is licensable under either a commercial or LGPL license. SciTech says the SNAP DDK will be released under a similar dual-license structure, but using the GPL rather than the LGPL. Additionally, SciTech has released the source to other related technologies, such as SciTech MGL and SciTech GLDirect. Due to the fact that nearly all of the chip level drivers were created under NDA (non-disclosure agreement) with hardware vendors, SciTech's individual drivers are generally only be available as closed source binaries. Should a hardware vendor wish to release open source drivers, SciTech says it can produce a driver to fit virtually any open source requirement.
The following further clarifications of SciTech's SNAP open source licensing are from the company's LGPL License FAQ:
- What is the difference between the LGPL and commercial tools? — The LGPL and commercial versions of our tools are identical code-wise. The LGPL versions of our tools are licensed for development of free or Open Source software. They include complete source code and are freely available under the GNU Lesser General Public License, or LGPL. You can use the LGPL versions of our tools to develop software free of charge provided that the software you develop is released under a license compatible with the LGPL and you follow the licensing requirements of the LGPL. If you develop commercial, closed source software, SciTech offers commercial licenses for this purpose. Commercial customers will benefit from professional support services and will not be restricted by the conditions imposed by the LGPL license.
- Why do you offer both a commercial and a LGPL version of your tools? — SciTech has put many years of effort and development into our professional products and development tools. Some developers may not be able to afford our tools financially, so the LGPL version of our tools allows those developers to pay for the use of those tools by making their software available to the free software community under a license compatible with the LGPL license. This allows our tools to be used by the largest target audience possible, both those developing free software under the various licenses and those developing closed source, proprietary products. The LGPL version of our tools also allows commercial developers interested in our tools to evaluate them free of charge prior to beginning commercial development.
- Are the LGPL versions of SciTech's tools free software? — Yes – both as in “free beer” and as in “free speech”.
- Can I build and design software with the LGPL versions of your tools and then release it under the BSD, Artistic or Mozilla licenses? — Yes. The LGPL license is compatible with many Open Source licenses. The LGPL is also compatible with many closed source development licenses, but requires you to allow the user to relink any LGPL code in your products (ie: either dynamically link to all the LGPL code or provide objects and makefiles so the user can relink your programs if the LGPL code is statically linked).
- Is software based on SciTech's LGPL tools really free? Does it carry SciTech license restrictions? — Yes, it is really free. No, there are no special SciTech license restrictions on free software produced using the LGPL tools.
- Using the LGPL Tools, can I make software for internal use in my company/organization? — SciTech's LGPL tools are not intended for such use; it is our policy that when you are using SciTech's tools for free, you should in return contribute to the free software community. If you cannot do that, you should get a commercial license from SciTech.
- Can I distribute applications that are dynamically linked to SciTech LGPL tools? — Yes, provided that your applications are also distributed with a full copy of the LGPL source code, or you provide a way for the customer to get that source code free of charge upon request. If you do not wish to provide this option, then you must license the commercial versions from SciTech. Putting the SciTech LGPL code into a library that is dynamically linked into your application does not mean you can avoid the licensing requirements of the LGPL license.
About the Authors: Kendall Bennett is CEO and Founder of SciTech Software Inc. Andrew Bloo is the Director of Sales and Marketing at SciTech Software Inc. Additional information about SciTech Software is available on SciTech's website.
This article was originally published on LinuxDevices.com and has been donated to the open source community by QuinStreet Inc. Please visit LinuxToday.com for up-to-date news and articles about Linux and open source.