Foreword — This article continues a series of tutorials on embedded Linux system development contributed by noted ARM Linux kernel hackers Vincent Sanders and Daniel Silverstone. In the initial installment, the Simtec engineers described how embedded devices differ from other computers, and how to build “the most basic system possible.” This second installment now covers the construction of a… simple web server.

Papers in the series have been contributed to LinuxDevices by Simtec Electronics, a 20-year-old U.K.-based company specializing in embedded hardware and software services, with special expertise in ARM Linux.

The series will include:

• Building a simple embedded system
• Building an embedded Linux system with a web server (this paper)
• Building an embedded Linux web kiosk
• Building an ARM-based embedded Linux web kiosk
• Improving an embedded Linux system
• Deploying embedded Linux systems

Enjoy . . . !

Introduction

This is the second article in a series demonstrating the fundamental aspects of constructing embedded systems.

In this article, we cover the construction of a simple web server with a command shell on the console.

This article, and indeed the whole series, assumes a basic understanding of a Linux-based operating system. While discussing concepts and general approaches these concepts are demonstrated with extensive practical examples. All the practical examples are based upon a Debian- or Ubuntu-based distribution.

Automated, reproducible, reliable building

One of the common pitfalls in building embedded systems is the tendency towards too much manual involvement in the build process. There seems to to be a misconception that because embedded systems are built, deployed and rarely updated that building them by hand saves time that would otherwise be spent automating the process.

Making the build process automatic and repeatable should be viewed as a critical part of the project. This enables the software engineers developing a product to have as short an edit, build and test cycle as possible. The desirability of a short development process should be self-evident, in that an engineer who can perform only one or two tests a day can only hope to debug and fix a small number of issues, where one who can perform a hundred tests can find and fix a far larger number of issues.

Another common mistake is not keeping the whole project in a Revision Control System (RCS). The benefits of revision control on any project have become increasingly evident and a wide selection of extremely powerful systems exist. The Subversion (svn) system is extremely popular for centralised RCS, while Bazaar (bzr) and GIT have become common for distributed RCS. Regardless of the model and tools chosen, revision control should never be omitted from a project.

For larger projects a centralized “build manager” is often desirable. This is a piece of software which builds the current project from the revision control system on a regular basis. Some projects rebuild on every commit, which may not be practical where a build takes an extended period of time. In such cases a system which rebuilds as often as it can, perhaps including numerous commits, should be employed. The results of these builds should be made available, and the developers informed as soon as possible of failures. This ensures the project is always in a state where it might be branched ready for formal testing and release.

A good article discussing these ideas further is Daily Builds Are Your Friend, by Joel Spolsky. Although this article refers to application development specifically, its analysis is valid on the larger project scale. Several other articles on best-practice for building software projects exist.

Scripting builds and common tasks

In the previous article we constructed an initramfs cpio-based system using the binaries of the host system. The steps were performed manually, and were we to continue with that approach any increase in complexity would rapidly make it impractical.

To solve this issue we turn to the Unix system's scripting tools. A shell script to perform the build actions makes the build easily repeatable and saves continually re-typing a lot of commands.

The mkbusyfs.sh [available here] script is an automation of the steps performed in the previous article.

In addition to the basic setup the script adds functions to copy executables and their library dependencies, configure a DHCP client and copy kernel modules from the host to the target. These functions are straightforward and self-contained and their operation should be obvious.

The script is written to be reusable for a number of projects by including a second "application" script. This enables us to reuse the base functionality in future articles. The scripts are provided under a BSD-style licence and may be taken and modified as desired.

To implement the simple system illustrated in the previous article, the simple.sh [available here] configuration script can be used.

To keep things neat these scripts should be placed in a directory (these examples assume that it will be called mkbusyfs) alongside where the output should be generated.

The simple system would be generated using the command ./mkbusyfs.sh simple and output would be placed in in the parent directory.

$ pwd
/home/dev/mkbusyfs
$ ./mkbusyfs.sh simple
Building simple
simple specific
Creating CPIO ../simple.gz
$ ls ..
mkbusyfs simple simple.gz
$

Creating a web server's file system

The first thing we must consider is which web server to install, this is of course dependant on our project requirements.

One choice might be the Apache Web Server, however this would be impractical for all but the largest embedded system. Apache's executables, library dependencies and other system dependencies are relatively large. A more suitable alternative would be thttpd which is a few hundred kilobytes and has very few dependencies.

First we need to create a mkbusyfs configuration script.

#!/bin/sh
# web server application specific mkbusyfs shell fragment
USE_DHCPC=y
EXTRA_LIBS="ld-linux.so.2 libnss_dns.so.2"
#KERNEL_VER=2.6.26-1-686
KERNEL_MODULES="kernel/drivers/net/ne2k-pci.ko 
kernel/drivers/net/8390.ko 
kernel/drivers/net/e1000e/e1000e.ko"
application_specific()
{
DESTDIR=$1
}

This configuration enables the DHCP client (which will require an init script, available here) and lists the kernel modules to be copied into the output. These are required drivers for network cards.

The project could be built and tested at this point if desired. It should initialise a system and acquire an IP address using DHCP.

Next the thttpd binary needs to be acquired. Instead of simply copying this from the host file system we can acquire the deb package from the package mirror, unpack it and extract the items we require without needing to install the package on the host. This does require the devscripts package to be installed but means no superuser privileges are required to build the system.

#!/bin/sh
# web server application specific mkbusyfs shell fragment
USE_DHCPC=y
EXTRA_LIBS="ld-linux.so.2 libnss_dns.so.2"
KERNEL_VER=2.6.26-1-686
KERNEL_MODULES="kernel/drivers/net/ne2k-pci.ko 
kernel/drivers/net/8390.ko 
kernel/drivers/net/e1000e/e1000e.ko"
application_specific()
{
DESTDIR=$1
CURDIR=$(pwd)
mkdir -p /tmp/thttpd/thttpd
cd /tmp/thttpd
dget thttpd
dpkg -x thttpd*.deb thttpd
cd ${CURDIR}
add_program /tmp/thttpd/thttpd/usr/sbin/thttpd /usr/sbin/thttpd
rm -rf /tmp/thttpd
}

If we were going to add a second package it might be worth extracting this package acquisition and unpacking into a function. Such judgements are of course arbitrary, but if something is done more than once it is often sensible to create a helper function as then, if a change must be made, all the affected uses will be updated.

The next step is to add the configuration to start the web server and add some basic content. The complete script can be downloaded here.

The only slight difference here is that an additional library and a /etc/passwd file was required so the web server could change to execute as the www-data user.

When the output is generated it may be tested using QEMU again. The command line to run QEMU is slightly different as it needs to enable a NIC and redirect the emulated systems port 80 to the host's port 8080. This allows the web server to be accessed from the host using a web browser and the URL http://localhost:8080/.

$ qemu -kernel ./vmlinuz-2.6.26-1-686 -initrd webserver.gz 
-append "root=/dev/ram" -net nic -net user 
-redir tcp:8080:10.0.2.15:80 /dev/zero

The pre-built Kernel and generated output for an x86 system are provided here and here.

What's next?

This second step demonstrates the ideas of automation and repeatability and shows how the basic environment constructed in the previous article can be expanded to produce a system capable of interacting with a user.

The next step is to use the concepts presented here and expand them by introducing a more complex application built from source and discussing some limitations of real hardware and issues that arise from it.

This article is copyrighted 2009, Simtec Electronics, and reproduced here with permission.

About the authors

Vincent Sanders is the senior software engineer at Simtec Electronics. He has extensive experience in the computer industry, having worked on projects from large fault-tolerant systems through accounting software to programmable logic systems. He is an active developer for numerous open source projects, including the Linux kernel, and is also a Debian developer.

Daniel Silverstone is a software engineer at Simtec Electronics, and has experience in architecting robust systems. He develops software for a large number of open source projects, contributes to the Linux kernel, and is both an Ubuntu and Debian developer.

 
This article was originally published on LinuxDevices.com and has been donated to the open source community by QuinStreet Inc. Please visit LinuxToday.com for up-to-date news and articles about Linux and open source.