Foreword: msystems is migrating its newest flash chips to open-source drivers. In this technical whitepaper, msystems provides an overview of three generations of flash technology, outlines the reliability challenges presented by the latest flash technologies, describes software techniques aimed at maximizing flash reliability, and introduces the soon-to-be-released open-source Linux drivers.

---

Opening the door for the latest NAND flash in open source mobile platforms

by Francois Kaplan

Spread the word: digg this story

Used mainly for code storage, NOR flash was once the prevailing non-volatile memory (NVM) technology in mobile handsets. But growing demand for multimedia applications including music download, cameras, gaming, and video has boosted demand for high-capacity, high-performance data storage that NOR cannot provide. More and more, NAND flash is replacing NOR flash as the most practical NVM solution for handsets. Over the last few years, NAND has become more compact and cost-effective at increasingly higher densities. Considering that storage is among the top three cost centers in a handset's bill of materials (BOM), cost-effectiveness is critical.

Advances such as MLC (multi-level cell) technology and smaller manufacturing geometries give NAND the clear cost advantage. But, as a result of these advances, the quality of NAND flash has degraded, making MLC NAND increasingly difficult to support. Without adequate management software, the latest, most advanced MLC NAND flash would be unusable in mobile handsets. But in the world of open source mobile platforms, introducing advanced management software means exposing sensitive IP to platform developers and handset vendors — a situation that MLC NAND flash vendors would rather avoid. This complication has made it difficult for the open source world to take advantage of the latest NAND flash.

In the first embedded flash drives (EFDs) that enabled access to high-capacity NAND flash in mobile handsets, the flash controller was integrated into the same device as the storage itself. This eliminated interface issues, providing a familiar NOR-like interface. But the flash management software still ran on the mobile device host.

First generation NAND EFD structure: flash management software running on the host
(Click image to enlarge)

In an effort to keep their highly sensitive competitive IP protected, flash vendors would insist on non-disclosure and licensing agreements for platform developers and handset manufacturers who wanted to integrate EFDs into their designs. For Linux and other open source platform developers, this simply would not do. Open source platform developers could instead integrate SLC NAND flash, which involves less IP. But, access to the latest MLC NAND — needed to support the latest storage-hungry, high-performance applications — would require breaking the “ethical code” of the open source community, making it difficult to maintain a competitive edge when working with open source software.

Then, along came the second generation EFD introduced by msystems: the mDOC H3. Adding to the existing advantages provided by first generation EFDs, such as providing both high-capacity data storage and bootable OS and application code on the same flash media, the second generation EFD introduced a new architecture. Second generation EFDs embed the flash management software within the storage device itself, running on the flash controller. With all of the advanced management commands hidden inside the EFD, integration is simpler and system performance improves by offloading flash management operations from the host CPU.

Second generation NAND EFD structure: embedded flash management software
(Click image to enlarge)

With sensitive IP now embedded in the storage device, flash vendors no longer need to worry about exposing competitive secrets. All that remains to integrate is the storage device driver, which is much thinner than the full array of flash management software, and which can be delivered as open code. In masking the sensitive IP to clear away the technology obstacles, the second generation EFD opens the door for open source mobile platform developers to access the latest MLC NAND flash technologies without sacrificing the open code vision. This makes open source even more attractive to the handset vendors.

Keep the IP and just give us the latest NAND

With more and more storage getting crammed into smaller and smaller spaces, developers can access the low-cost, high-capacity storage they need at consistently dropping prices. But what about system reliability? What about system read/write performance? How easy is NAND flash to integrate?

Okay. So NAND is not perfect. Actually, as a raw material, NAND flash comes out of the fab with a lot of problems that can only be corrected with up-to-date flash management software. Below is a sampling of these problems.

• Physical vs. logical mapping — Flash media is organized in physical blocks, or erase units, which are further divided into physical sectors. Standard file system calls specifying hard disk sector numbers and cylinders cannot be used to access data on the flash media. Dynamic virtual mapping is required, in order to map the file system model to the flash physical model.
• Time-consuming and life-reducing P/E cycle — To write, or program new data, existing data must first be erased. The process of erasing and programming is known as the P/E cycle. Each flash sector can withstand a limited number of P/E cycles before it becomes unusable. When this happens, overall capacity is gradually reduced, existing data becomes prone to random errors and application and OS code can quickly become corrupted, increasing the likelihood of total application failure. As manufacturing geometries shrink and the density of flash media increases, more complex error checking and re-writing is required. This further reduces the usable lifespan of the media, degrades performance and increases power consumption. Advanced dynamic and static wear leveling is required to extend the lifespan of the media, especially when it stores critical data such as OS and configuration files.
• Randomly scattered bad blocks — MLC NAND media are knowingly shipped from the fab, with up to 5 percent randomly scattered bad blocks. In addition to the “initial” bad blocks, normal wear from writing and erasing cause additional blocks to become corrupted, and eventually unusable. Bad blocks need to be located and tracked, so that they are avoided during write operations.
• Bit flipping and bit pairing — A bit can reverse its charge or be reported as reversed. If even a single bit reverses in a block that stores boot code, OS, or configuration files, the reliability of the device could be compromised. In MLC NAND, two or more bits are stored in each physical cell of the flash media. If a single bit is corrupted (by a power disturbance or software error during a write operation) it corrupts its paired bit in the cell, even if the paired bit had already been written successfully. What's worse, bit-pairing schemes differ between NAND flash vendors, and even between different NAND generations from the same vendor. Innovative error detection code (EDC) and error correction code (ECC) must be used to ensure data reliability without negatively impacting performance. EDC and ECC need to take into account the different types of NAND flash errors (bit flipping, bit pairing, charge drift, etc.) and the different ways that they manifest from vendor to vendor.
• Data retention errors — Over time, current leakage or charge drift can slowly change a cell's voltage level, which could incorrectly be interpreted as a different logical value. This impacts data retention, which is a measure of how long data can be stored without deteriorating to the point where error detection and correction can no longer correct the errors.
• Power failure immunity — In flash memory, the P/E cycle uses “erase before write” algorithms, which can lead to longer write cycles more likely to be interrupted during power failures, leading to lost or corrupted data and code. Advanced flash management technologies implement “erase after write” algorithms to ensure data integrity during normal operation and in the event of a power failure.

MLC NAND flash technology — more trouble

Increasing NAND flash bits per cell density reduces cost and size, but exacerbates flash limitations, degrading reliability, performance and flash lifespan — further complicating ease-of-integration. In flash devices that implement standard Single Level Cell (SLC) technology, one bit of data is stored per cell, using two voltage levels. Multi-Level Cell (MLC) stores two bits of data per cell, using four voltage levels. MLC is the most advanced, cost-effective NAND technology to date. But, it also presents unsurpassed complexity, requiring far more complex management technologies.

Voltage level comparison between SLC and MLC flash technologies
(Click image to enlarge)

Making the latest NAND flash easy-to-use for open source platforms

To solve the increasingly complex problems of managing MLC NAND flash, second generation EFDs deliver all of the necessary technology wrapped in one package together with the flash media and the controller. With only a thin device driver exposed as open source code, and all of the complex IP embedded inside the EFD, the flash device appears to the operating system as a standard, virtually plug-and-play, high-performance drive. And it also delivers a solution for open source mobile platforms that conforms to the principles and the exacting requirements of open source development.

Second generation EFD mDOC H3 and thin DOC driver
(Click image to enlarge)

Putting together all the pieces… almost

Demand for on-board, high-capacity storage will continue to rise in the mobile marketplace — and demand for MLC NAND flash, the most cost-effective solution for mobile handsets, will rise with it. Managing MLC NAND is still a Herculean task, presenting complex problems that new second generation embedded flash drives transparently solve to ensure high performance, high reliability, and long data retention and flash lifespan. What's more, by masking the management complexity, EFDs also mask the flash vendor's IP.

By delivering the flash, controller, and flash management IP all together in a single, closed package, second generation EFDs such as the mDOC H3 from msystems leave only the simple device driver exposed as open source for easy integration free of the IP legal hassles of the past.

Transparent management and IP, by eliminating technical and legal challenges, let open source mobile platform developers access and support the latest, most reliable, high-performance MLC NAND flash at the capacities demanded by today's most popular applications. With the related development delays and other storage related obstacles removed, the most advanced designs with the most advanced features — running on Linux and other open source platforms — can get to market more quickly, without sacrificing the vision and ethics of the open source community.

---

About the author: Francois Kaplan, Associate Vice President of Product Marketing for msystems's mobile division, manages worldwide product marketing activities for both customers and partners. Previously, he served as General Manager of msystems Europe. In this capacity, he was in charge of tier one customers, analyzing their needs and concerns to help drive product design. His activities were instrumental in the first implementation of NAND-based solutions by mobile OEMs. Kaplan has held numerous management, sales, marketing and engineering positions for world-class companies, including IBM France. He earned an engineering degree from Ecole Centrale de Lyon, a top school in France. Kaplan can be reached via email.

msystems has been a pioneer in flash storage solutions since 1989. The company's flash devices have been used countless applications in a wide range of markets, including millions of mobile handsets.

---

 
This article was originally published on LinuxDevices.com and has been donated to the open source community by QuinStreet Inc. Please visit LinuxToday.com for up-to-date news and articles about Linux and open source.