Spotlight on Carrier Grade Linux — Open Standards, Solutions, Compliance

Dec 7, 2004 — by LinuxDevices Staff — from the LinuxDevices Archive — views

Foreword: This whitepaper from TimeSys discusses Carrier Grade Linux (CGL), and changes in the CGL 2.0 requirements document released last fall. It also discusses CGL 2.0's self-registration process for vendor compliance, which TimeSys target=”new”>completed first, among CGL vendors. Enjoy . . . !

Introduction

The phenomenal growth of the Internet, the proliferation of personal communication devices, and the exponentially decreasing cost of hardware have resulted in a rapid expansion of telecommunications and networking infrastructure, and have facilitated the development of many new telecommunications and data services. These changes, and the increasing globalization of businesses, have led to an unprecedented need for connectivity.

Until recently, the primary systems that were considered mission critical were data center and carrier switching systems. The need for always-on connectively is now pervasive across most information technology and telecommunications infrastructure. Increasing requirements for connectivity, availability, and reliability have come at the same time that the information technology, telecommunications, and networking industries are faced with competitive pressures that require them to innovate at a rapid pace, and at the same time cut operating costs. All of this has led to a shift in how new platforms, systems, and devices are designed, implemented, brought into production, and sustained after deployment.

In the telecommunications and networking industries, these market forces have resulted in a move from proprietary approaches in carrier grade systems design to an approach based on standardized, commercial off-the-shelf (COTS) single-board computers, data collection, and interface boards. The increasing reliance on COTS in system design has also led to changes in buyer, supplier, and competitor relationships, making them more collaborative, where each brings their core competencies to enhance the value of the offering provided by others. This collaboration helps drive standardization efforts across the entire ecosystem, thus ensuring that the COTS movement maintains momentum and continues to deliver lower costs in future systems.

Hardware Standardization

While the telecommunications industry has a long history of developing standards, the move towards COTS components has driven new hardware standardization efforts. In recent years, the telecom industry has collaboratively developed and adopted standards such those that make up the CompactPCI (CPCI) Specification and the Advanced Telecom Computing Architecture (AdvancedTCA or simply ATCA), both supported by the PCI Industrial Computer Manufacturer's Group (PICMG).

Compact peripheral component interconnect (CPCI) is an adaptation of the peripheral component interconnect (PCI) specification used in most desktop computer systems today. CPCI was developed in 1994 to meet the needs of industrial computer applications requiring a stronger, more rugged mechanical form factor than that used in desktop systems. CPCI uses the Eurocard form factor (3U and 6U) first popularized by VME, and supports twice as many PCI slots (eight) on a single bus as does standard PCI. It provides integrated support for live insertion and removal, standardized administration through protocols such as the Simple Network Management Protocol (SNMP), and provides the traditional data routing and interconnect advantages of any backplane/chassis solution.

The Advanced Telecom Computing Architecture (AdvancedTCA or simply ATCA) is a new emerging standard that builds upon the experience gained adapting CPCI to the needs of telecommunications and network equipment providers. ATCA focuses on providing a high-performance, more-scalable backplane environment that supports higher levels of throughput, interconnection, and better system management facilities, serviceability, and scalability. ATCA mandates a standard management framework (built on IPMI 1.5), star and full-mesh interconnect mechanisms, and robustness and availability features such as redundant power connections to each board. For additional ATCA information, see Web sites such as Intel's ATCA page, Motorola, and Freescale.

Software Standardization

Hardware standards that guarantee compatibility between offerings from multiple vendors are most effective when complemented by the development of software standards that define operating system capabilities and mechanisms for efficiently using and managing standard hardware. Over the years, software standards for programming languages, operating system interfaces, and application middleware have enhanced application portability and interoperability, reducing reliance on any single vendor solution.

The Carrier Grade Linux (CGL) specification, developed under the stewardship of the Open Source Development Laboratory (OSDL), exemplifies this trend. CGL was developed in collaboration with the Service Availability Forum, the Free Standards Group, multiple Linux vendors, and telecommunications and networking firms. CGL compliance provides interoperability and capability guarantees for Linux distributions targeted at the telecommunications and networking markets. The CGL specification defines the software platform capabilities that developers can rely on in order to build reliable, available, and serviceable software solutions that take full advantage of the capabilities provided by hardware specifications such as CPCI and ATCA.

Hardware and software standards developed in an open environment push for the betterment and requirements of an industry rather than the benefit of any particular vendor. As discussed in more detail later in this white paper, compliance with CGL 2.0 and beyond is demonstrated by openly publishing details about how each mandatory requirement was satisfied. This preserves the spirit of an open specification between software partners, while enabling customers to see proof of compliance. Open demonstrations of compliance also enable CGL users to do an independent technical assessment of a particular CGL implementation, to verify its suitability for their needs.

Focus Areas for Carrier Grade Linux

Carrier Grade is a telecommunications and networking industry term for systems that must be available 99.999 percent of the time, known as “5 nines.” This level of availability implies that “5 nines” compliant products do not require more than 5 minutes of downtime per year. The “Carrier Grade” label imposes even stricter requirements for minimal downtime and high availability than the more general information technology term “24×7,” which means continuous operation, 24 hours a day and 7 days a week.

The OSDL's CGL specification defines a standard set of requirements that satisfy the needs of the telecommunications and networking industries, and that provide a common base for developing interoperable software. In conjunction with related telecommunications and networking hardware standards, the CGL specification provides a firm foundation for the development of standard, high-performance, highly-available, scalable, and easily-serviced platforms.

The CGL specification groups requirements into the categories shown in the following table, listed alphabetically.

Requirement	Definition
Availability	Capabilities associated with high availability, such as reducing single points of failure, resource failover, multipath I/O, fast restart, the ability to force and remotely manage system reboots and upgrades, and so on.
Clustering	Requirements for using multiple, connected, carrier grade systems as a single computing resource in order to provide higher levels of service, increased scalability, improved performance, and increased uptime.
Performance	Features required to satisfy responsiveness and general performance goals. These include soft real-time support, constant-time scheduling, process priority inheritance, processor affinity, specific mechanisms for handling flexible amounts of transient data, and so on.
Platform	Hardware-related standards and supported interfaces that must be provided by a CGL Linux distribution in order to work with standard telecommunications and network equipment provider standards such as ATCA.
Scalability	Requirements necessary to support carrier-class server systems where adding hardware resources and associated software configuration metadata produces appropriate increases in performance and capacity.
Security	Facilities to protect and ensure the consistency of carrier grade systems and the data that they contain.
Serviceability	Capabilities associated with remote administration and management of systems running a CGL distribution.
Standards	The core set of services and capabilities that must be available in a CGL Linux distribution. These include references to POSIX, IEEE, ISO, and Linux standards such as the Linux Standard Base (LSB) runtime compatibility specification, which in turn reference other Linux standards such as the Filesystem Hierarchy Standard (FHS).
Tools	Requirements for tools that support application and system development, debugging, and performance analysis, including tools that enable analysis and debugging for deployed systems and applications.

The requirements that comprise each of the sections of the CGL 2.0 specification are prioritized into one of three different levels. These are:

Requirement is mandatory and must be satisfied in a CGL system in order for it to be compliant with the specification.
Requirement is desirable but not mandatory.
Requirement is identified as relevant to some CGL systems but is not currently viewed as critical

As discussed in more detail later in this white paper, CGL 2.0 Linux distributions must demonstrate that they have met all priority 1 requirements, or provide a technical explanation for any non-compliance. These CGL registrations must be published openly, and can therefore be readily examined by any potential user of that distribution.

The Evolution of the CGL Specification

The CGL specification has gone through several official revisions and releases, and is currently evolving again. The first major release of the CGL specification was Version 1.0, released in August 2002. This was followed relatively quickly by CGL 1.1 in September 2002. CGL 2.0 was finalized in October 2003, primarily adding clustering and security requirements to the 1.1 specification, and modifying the priority of some existing items. Work on a public draft of the CGL 3.0 specification is currently in progress, and is expected to be completed in 2005.

For the most part, later versions of the CGL specification are a superset of previous versions. While the broad themes of the CGL specification have remained unchanged, the priorities of individual requirements may change across different versions of the specification. This is typically driven by factors such as the increasing maturity of a particular implementation of the requirement, mainstream acceptance of a particular technology or implementation, perceived importance, and so on. For example, support for both SCTP, the Stream Control Transport Protocol, and Remote Installation and Upgrade Support were priority 2 in the 1.1 specification, but are priority 1 in the 2.0 specification.

While the general trend is to increase priorities, this is not always the case. The lack of a mature implementation of a particular capability, too much ambiguity in the requirement definition, or the emergence of new, more critical requirements may lessen the priority of certain requirements between consecutive versions of the CGL specification. Some requirements identified as priority 1 in the 1.1 specification, such as Device Driver Hardening Specification, a Resource Monitor Specification, were not brought forward into the CGL 2.0 specification. Some lower priority items from the 1.1 specification, such as debugging support for fork(), requirements related to High Concurrent Timer Scaling, kernel I/O performance analysis, and so on, were also dropped from the 2.0 specification.

The most significant way in which the 2.0 CGL specification differs from the 1.1 specification is the addition of detailed clustering and security requirements. Clustering is an inexpensive way to create reliable, recoverable, and fault-tolerant systems, by combining a highly integrated group of interoperable machines into a single, flexible computing resource known as a cluster. Clusters can dynamically respond to topological changes or the failure of any single resource within the cluster. Similarly, a cluster provides an inexpensive way to increase performance by taking advantage of distributed processing, computation, or even available computational cycles on an otherwise underused node in the cluster. Clustering technologies for Linux such as Beowolf, Scyld, and openMosix, have been in use for years, and no single solution for creating and managing Linux clusters has emerged as the “one true clustering solution.” For this reason, and because clustering computing and data-sharing techniques continue to improve and stabilize, none of the clustering requirements in the CGL 2.0 specification are identified as priority 1 (mandatory) requirements.

As mentioned previously, maintaining and extending a living standard requires striking a good balance between wish lists, critical features, and the realities of implementation, integration, and support.

CGL and Standard Linux Distributions

A common question from interested parties is how CGL-related requirements and enhancements affect a “standard” Linux distribution. Is it still Linux? Is the source still freely available? Where do the enhancements required for CGL registration actually come from? Providing an overview of the Linux kernel development and enhancement process provides good insights into the answers to these and related questions.

Linux is the operating system of choice for most new embedded system development projects. Embedded Linux vendors such as TimeSys produce hardware-optimized Linux distributions that are tailored to specific custom and reference embedded hardware platforms. The release of the 2.6 Linux kernel has increased the general performance and responsiveness of Linux, the number of supported processors and architectures, and the spectrum of supported buses, interfaces, and devices. 2.6-based Linux distributions are extremely stable, are available from almost every major Linux vendor, and provide substantial technical improvements over earlier versions of the Linux kernel, such as 2.4.

Parallel to mainline development efforts to increase the performance and hardware support in Linux are numerous software projects focused on improving and extending its core capabilities. The development of most of these projects takes place outside of the core kernel development line, with the hope that many of them will eventually be merged into the kernel. Whether projects are eventually merged into the kernel depends on a number of factors such as whether they are useful to the general Linux community, whether they are cleanly and “correctly” implemented, and whether they are philosophically viewed as “the right thing” to be integrated into the kernel.

The standard Linux kernel is designed to be a general-purpose, high-performance kernel, with no particular bias towards the needs of any one industry, set of users, or hardware architecture. Some of the capabilities required by industry-focused specifications like CGL, such as integrated kernel debugging for serviceability purposes, may never be merged into the core kernel, because of their potential impact on overall Linux performance, or because there is no single, architecture-neutral solution that stands out between competing approaches to solving that particular problem.

For example, many of the capabilities required by specific industries are already available for Linux, but have not yet been merged into the kernel for one reason or another. The CGL specification is a good example of this, because it identifies a number of requirements in areas such as availability, serviceability, and scalability that are not currently satisfied by the vanilla Linux kernel available from kernel.org. This is not a deficiency in the standard Linux kernel, but a simple by-product of different focus areas.

Processor and architecture issues require special consideration when developing and supporting a Carrier Grade-registered Linux distribution. The x86 architecture and related platforms are the traditional focus area for Linux development, and many of the Open Source projects that provide the features required for CGL registration have been developed and tested only on x86 platforms. However, most telecommunications and network industry infrastructure is based around PowerPC (PPC) processors and related platforms. To serve the CGL market, vendors such as TimeSys who are producing CGL distributions must have historical expertise in developing Linux distributions for segments where PPC processors have also always had primary market share, such as embedded computing. Such vendors are therefore extremely familiar with architecture-specific issues associated with porting and supporting high-performance Linux on non-x86 platforms.

CGL Registration: Demonstrating CGL Compliance

The CGL specification differs from traditional specifications in that it defines a set of mandatory capabilities but does not restrict their implementation. In contrast to traditional API-based specifications that rigidly define programming interfaces, capability specifications such as CGL provide the flexibility necessary in the Open Source community to select the best technical solutions from multiple projects that share similar goals. Demonstrating compliance with a specification that defines high-level capabilities is a non-trivial task, because no simple API test can be developed. Demonstrating compliance with an open, dynamic specification requires flexibility, as opposed to the pass/fail conformance tests that are associated with API-based standards.

The key to demonstrating compliance with a capability specification is to identify the kernel patches, system applications, and user-space utilities that have been integrated into a distribution in order for it to fulfill the requirements of the specification. While the OSDL has choreographed the development and evolution of the CGL specification, it is not their responsibility to hand out “CGL Compliant” stickers that can be pasted on the boxes of Linux distributions that claim CGL conformance. For this reason, the OSDL and other participants creating the CGL specification came up with the idea of “CGL Registration,” which offloads the responsibility of proof of CGL compliance to the vendor, in a public fashion that can easily be examined by any potential customer who is interested in a CGL-compliant Linux distribution.

TimeSys has worked closely with the OSDL to create a mechanism for openly demonstrating compliance with the CGL specification. Prior to version 2.0 of the CGL specification, CGL distributions simply declared compliance. It was up to the consumer to determine the degree and extent of compliance, and the technical merits of capability implementations.

With the CGL 2.0 specification, the OSDL introduced the notion of registered CGL distributions that can be instantly verified by viewing an online XML registration document. This XML document is formatted according to the rules defined by an open XML document type definition that is available to all potential CGL distributions, and which provides specific ground rules for documenting compliance to mandatory components of a CGL specification. Documentation on CGL registration requirements is available in a downloadable PDF. As an example of a document that conforms to this format and demonstrates compliance with the CGL specification, TimeSys's CGL registration document is available here.

CGL registration documents require various types of information, such as the source of the patches and additional applications required to demonstrate CGL compliance at both the kernel and filesystem level. As documents that show conformance to an open specification, CGL registration documents also provide a document element designed to explain where a vendor's CGL implementation may not be completely conformant to the specification. Initially, explaining non-compliance in a compliant document seems counter-intuitive, but the reality is that some requirements are flexible or large enough that they cannot be easily defined in a single document.

A good example of this is conformance to the Linux Standard Base specification, which defines mandatory applications that must be present in the filesystem of a compliant Linux distribution, defines their location, and provides runtime compatibility for compliant applications. While defined as a single CGL requirement, demonstrating compliance with this item could require thousands of lines of information that is essentially meaningless to readers. As such, items such as this are demonstrated by providing links to relevant, external documents, and the date on which those compliance documents were submitted.

By defining a specific document format that contains pointers to the source code used to add required features, and by making these documents publicly available, registering a Linux distribution that is compliant with the CGL specification is conformant with both the spirit and letter of an Open Source specification. Any vendor that wishes to register their distribution as CGL-compliant makes their additions and implementation highly visible and freely available. Though this may be anathema to proprietary operating systems and closely-held, customized Linux distributions, this is exactly “the right thing” for a truly open, cooperative specification.

Summary

The Carrier Grade Linux 2.0 specification represents a tremendous step forward for the telecommunications, networking, and data delivery industries in the move towards low cost, COTS-based platforms. Concurrent with evolving hardware standards such as CPCI and ATCA, defining a standard set of operating system, software development, and user-space capabilities facilitates the development of interoperable, portable, and maintainable systems and applications. Linux distributions that can demonstrate compliance with the latest, up-to-date CGL specification, such as TimeSys' CGL 2.0-compliant distributions, provide a firm foundation for the development of next-generation COTS-based telecommunications and networking platforms.

Regardless of the CGL-compliant distribution that you select, the emergence of Linux software standards supported by multiple vendors and the ability to openly demonstrate how a distribution achieves compliance is a critical step forward for Linux, and provides significant benefits for customers who require CGL-compliance, for the vendors who provide these distributions, and for the Open Source movement in general.

More Information

TimeSys offers a number of other white papers on related subjects such as the 2.6 Linux kernel, tools for developing embedded applications, and real-time Linux. For additional white papers and other relevant information, please visit TimeSys. TimeSys also offers webinars on various topics.

About the author

William von Hagen is a Senior Product Manager at TimeSys Corp., has been a Unix devotee for over twenty years, and has been a Linux fanatic since the early 1990s. He has worked as a system administrator, writer, developer, systems programmer, drummer, and product and content manager. Bill is the author of Linux Filesystems, Hacking the TiVo, SGML for Dummies, Installing Red Hat Linux 7, and is the coauthor of The Definitive Guide to GCC (with Kurt Wall) and The Mac OS X Power Users Guide (with Brian Profitt). Linux Filesystems is available in English, Spanish, Polish, and traditional Chinese. Bill has also written for publications including Linux Magazine, Mac Tech, Linux Format, and online sites such as Linux Planet and Linux Today. An avid computer collector specializing in workstations, he owns more than 200 computer systems.

Read the entire series . .

Previous whitepapers by William von Hagen:

TimeSys also offers webinars on a variety of topics.

This article was originally published on LinuxDevices.com and has been donated to the open source community by QuinStreet Inc. Please visit LinuxToday.com for up-to-date news and articles about Linux and open source.

Comments are closed.

Pages

Archives

Categories

Spotlight on Carrier Grade Linux — Open Standards, Solutions, Compliance

Related Posts: