ELJonline: Booting from the Ether
Sep 1, 2001 — by LinuxDevices Staff — from the LinuxDevices Archive — 1 viewsOne of our finalists shows you how to enable the NIC to boot from the network.
EtherNET that is. I first read about the ThinkNIC New Internet Computer in Linux Journal, and I ordered one as soon as I finished the article! I had so many plans for the unit, none of which had anything to do with its original intention. If I could get my grandma to make room for one in her house, she could use it the way the manufacturer intended, but that's another story.
After booting the unit from a CD, I copied the CD to my development Linux box and started playing around. I made some changes then made a new CD. Of course I also made a typo, so I had to throw that CD away and make another one. This one brought up a shell window in addition to the Netscape window when the system started. That was fun, but a slow and expensive way to develop. I needed a way to make changes without having to burn a new CD each time.
The ThinkNIC computer comes with a 10/100 Base-T Ethernet adaptor based on the SiS 90010 chip. It even includes a boot ROM that allows you to boot the unit from the network. This article will describe the steps necessary to do this.
Booting a PC 101
Before we start talking about booting from the network, let's go over some basics on how a computer boots. A lot of things happen when you turn on a PC.
After all the hardware tests successfully, the system will check for other BIOS ROMs in the system. If a hardware manufacturer created a PC card that needs to be initialized when the system turns on, they could put a ROM on the card that the system BIOS could call to initialize the piece of hardware. If you have an Adaptec SCSI card in your system, you've probably seen the list of SCSI devices when your system boots. This list is printed by the SCSI card's BIOS ROM.
It is now time for the BIOS to look for a device that has an operating system. When you configure the BIOS, you usually can specify where to look for an operating system. By default, the system looks on the diskette in the first floppy drive, then on the first IDE hard drive. Usually, you can configure your system to look on the CD-ROM drive as well as on an SCSI drive. On the ThinkNIC computer, you can tell it to attempt to boot from the network.
The system BIOS loads one sector worth of information from the boot device and runs the program contained in that sector. The program code in that sector needs to load the rest of the operating system.
Booting from the Network
The boot ROM in the SiS 90010 chip implements the PXE boot protocol. PXE stands for preboot execution environment. It supports networking standards such as TCP/IP, TFTP and DHCP, so it should be simple to use with a Linux server. There will be a lot of acronyms in this section, so see the glossary for definitions.
The PXE boot ROM first sends a DHCP request to get an IP address. The DHCP server can send more information, such as the location of the boot-loader software. If the DHCP server doesn't send the location of the boot-loader software, the PXE ROM will send out a request for a PXE server.
Assuming the DHCP server sends the location of the boot-loader software, the PXE ROM will use TFTP to retrieve this software. The software will retrieve a Linux kernel that will run on the ThinkNIC computer. This kernel should be configured to use NFS to mount its root filesystem across the network.
Setting up the ThinkNIC
You will need to tell your ThinkNIC BIOS to boot from the network. When you turn on your computer and see the message “Please wait, your NIC is starting”, press the Delete key on your keyboard. This should bring up the BIOS CMOS setup utility. Select the BIOS FEATURES SETUP option with the arrow keys and press Enter. Use the down arrow key to highlight the Boot From ROM Loader option. Use the Page Down key to set this to Enabled. Press the Escape key to leave this screen, then press F10 to save your changes and exit. If you have a CD-ROM in your ThinkNIC computer, remove it now.
When the ThinkNIC boots, you will see a message that says:
Intel UNDI, PXE-2.0 (build 068)
Copyright (C) 1997,1998,1999 Intel Corporation
DHCP MAC ADDR: XX XX XX XX XX XX
DHCP..
Write down the numbers next to MAC ADDR; these numbers represent the Ethernet hardware address of your network board. Sometimes this is called the MAC address of the card. Every Ethernet card in the world should have a unique Ethernet MAC address. We need to tell the DHCP server to give out a specific Ethernet address when it sees a request from this hardware address.
Installing a DHCP Server
My home network has a router that functions as a DHCP server and is connected to a cable modem. I was worried about having two DHCP servers on my home network, but so far this hasn't caused any problems.
Now you should set up a DHCP server on a Linux system on your network; I'm using Mandrake Linux 7.2. The name of the RPM to load for this distribution is dhcp. It might be called dhcpd on some distributions.
The configuration file for the DHCP server on my computer is /etc/dhcpd.conf. Below is a listing of my /etc/dhcpd.conf file. The IP addresses have been changed to protect the innocent:
subnet 192.168.1.0 netmask 255.255.255.0 {
}
host nic {
hardware ethernet 00:e0:06:01:01:01;
fixed-address 192.168.1.12;
option subnet-mask 255.255.255.0;
option broadcast-address 192.168.1.255;
option routers 192.168.1.1;
filename "/bpbatch/bpbatch";
}
You should put the MAC address of your ThinkNIC next to hardware Ethernet in the file. Note that each two-digit hexadecimal number is separated by a colon. You should also make sure all the IP addresses are correct for your local network.
You need to start your DHCP server. The Mandrake Linux command to do this is /etc/rc.d/init.d/dhcpd start.
Now, when you boot your ThinkNIC computer, you should see the following on screen:
Intel UNDI, PXE-2.0 (build 068)
Copyright (C) 1997,1998,1999 Intel Corporation
DHCPD MAC ADDR: 00 E0 06 01 01 01
CLIENT IP: 192.168.1.12 MASK: 255.255.255.0
DHCP IP: 192.168.1.x
GATEWAY IP: 192.168.1.1
TFTP.
PXE-T01: File not found
TFTP.
PXE-T01: File not found
PXE-E3B: TFTP Error - File Not found
Please insert boot CD into CDROM drive then
Press Enter ...
There are plenty of errors, but this is great news overall. It tells you that the ThinkNIC was able to get its IP address, but it wasn't able to load a boot loader. And it shouldn't have been able to load a boot loader, as we haven't set that up yet. If it gives the error message PXE-E51: No DHCP or BOOTP offers received, you should make sure you have started the DHCP server and connected the Ethernet connector on your ThinkNIC. If you received the error message PXE-E53: No boot filename received, the ThinkNIC received an IP address but wasn't told which boot-loader filename to load.
Install a TFTP Server
TFTP (trivial file transfer protocol) server is the protocol that the boot ROM uses to download the boot-loader software. It is similar to FTP but has no security built into the protocol. If you are concerned about others accessing the files you have on your TFTP server, make sure to use TCP wrappers to protect it. I won't go into details on how to do that here; there are many documents that explain it.
The Mandrake Linux RPM package that contains the TFTP server is called tftp-server. Make sure this package is installed on the same server that has your DHCP server. You can make it work spread over different servers, but it's much easier if everything is on the same system. Install the TFTP server package and create the directory /tftpboot. Give it world-readable permissions.
This TFTP server isn't a dæmon that constantly stays in memory. It is run from the inetd dæmon as needed. If you are using inetd on your computer, make sure your /etc/inetd.conf file has the following line in it:
tftp dgram upd wait root /usr/sbin/tcpd in.tftpd -s /tftpboot
If you are using the newer xinetd dæmon (as Mandrake Linux 7.2 does), make sure the /etc/xinetd.d/tftp file on your system contains the following:
service tftp
{
socket_type = dgram
wait = yes
user = root
log_on_success += USER
log_on_failure += USER
server = /usr/sbin/in.tftpd
server_args = -s /tftpboot
disable = no
}
The important thing is the -s /tftpboot argument because it tells the TFTP server to do a chroot command into the /tftpboot directory. This will protect files on your server because by default, the TFTP server will allow downloads of any world-readable file. When you use this argument, the TFTP server will only allow downloads of world-readable files in the /tftpboot directory.
If you have made changes to any of these files, you will need to restart your inetd or xinetd dæmon. Use this command for the inetd dæmon:
/etc/rc.d/init.d/inetd restart
Use this command for the xinetd dæmon:
/etc/rc.d/init.d/xinetd restart
Install Boot-Loader Software
When I got this far, I decided to put a Linux kernel on the TFTP server for the ThinkNIC system to load. Unfortunately, that didn't work. It seems there is a limit to the size of the file that the ROM will retrieve. I've seen 32K mentioned in newsgroups, but I still haven't seen the specifications for PXE, so I'm not positive if 32K is the limit. As it would be difficult to squeeze a Linux kernel into 32K, I had to look for alternatives. I found out that GNU GRUB (Grand Unified Bootloader) will work with a PXE boot ROM. Unfortunately, it doesn't have a driver for the Ethernet card in the ThinkNIC. GRUB uses the Ethernet drivers from the Etherboot package. Etherboot 5.0.1 supports the card, but GRUB uses the Etherboot 4.5 drivers. So I tried to make GRUB use the new Ethernet drivers, but after spending three or four hours on it, I wasn't successful. If someone is looking for a way to help us all out, it shouldn't be terribly difficult to update the drivers in GRUB.
Another boot-loading program is a program called BpBatch. It is licensed for personal use only. If you want this for your business, you'll have to contact them for pricing. (You could work on GRUB instead so everyone wins.) I'll show you how to use BpBatch, but then I'll give you another, free alternative at the end of the article.
Download a copy of the BpBatch program (see Resources). Be careful when untarring the file as it does not untar into a new subdirectory. This is a pet peeve of mine–all programs should untar into a new subdirectory.
Make a bpbatch subdirectory under the /tftpboot directory. Remember, everything under this directory should be world-readable. Copy the bpbatch.hlp and bpbatch.ovl files into the /tftpboot/bpbatch directory. Rename the bpbatch.P file to bpbatch, and copy it into the /tftpboot/bpbatch directory also. You can read the documentation for an explanation of why we renamed the file. Now we need a script to tell BpBatch which kernel to load and what other parameters are needed. Here are the contents of the file /tftpboot/bpbatch/bpbatch.bpb:
Set CacheNever = "ON"
Set CacheAlways = "OFF"
linuxboot "/bpbatch/bzImage" "rw root=nfs
nfsroot=192.168.1.5:/nfsroot/nicroot
ip=192.168.1.12:192.168.1.5:192.168.1.1 :255.255.255.0::eth0"
The value of the nfsroot parameter is the IP address of your server and the path to the root filesystem that we will create later. The nfsaddrs parameter has seven parameters separated by colons. The values are the IP address of your ThinkNIC system, the IP address of your server, the IP address of your network router, the subnet mask of your network, the hostname, the device you are configuring and the protocol you are using to get your IP address.
If you boot your ThinkNIC computer now, it should load the BpBatch program and the script but fail when trying to load the Linux kernel. If you have problems, make sure your DHCP and TFTP servers are running and your configuration files are correct.
A ThinkNIC Kernel
We need to create a kernel for the ThinkNIC. I haven't been able to build a kernel and a filesystem that duplicates the functionality of the CD-ROM. Instead I went ahead and compiled my own kernel and created my own filesystem.
I have been using a 2.4 version kernel on my ThinkNIC. This version has USB support and seems to do everything I need. The ThinkNIC comes with the 2.2.15 kernel with a special patch to support USB. You can pick any version you like as long as it supports the SiS 900 Ethernet chip. You'll need to make sure to include all the options necessary for mounting the root filesystem via NFS. Also, these options shouldn't be compiled as modules. The code for these options needs to run before the root filesystem is mounted, so modules can't be loaded at that time.
When you build your kernel, make sure to include at least these options:
- Network Options/IP: kernel level auto-configuration
- Network Device Support/Ethernet/EISA, VLB, PCI/SiS 900/7016
- Filesystems/Network Filesystems/NFS filesystem support
- Filesystems/Network Filesystems/Root filesystem on NFS
Use the make bzImage command to build a compressed kernel image. The file will be placed in the arch/i386/boot/bzImage file under your Linux source code directory. Copy this file to the /tftpboot/bpbatch directory. We're still not done, but you can try booting your ThinkNIC computer again. It should boot, load the kernel, then tell you that it was unable to mount the root filesystem.
Load the ThinkNIC Filesystem on the Server
We need to create a root filesystem on the server for our ThinkNIC. I wouldn't recommend using the exact same root filesystem that you use for your server. The root filesystem contains system configuration files, and it is almost certain that your server needs a different configuration than your ThinkNIC system. At the least, the IP address needs to be different.
For this article we'll use the contents of the ThinkNIC CD-ROM for our root filesystem, but you aren't limited to that. I have used a Mandrake Linux filesystem on the ThinkNIC with great success. I have also built a filesystem from scratch. It all depends on your application for the box.
Make the directory /nfsroot/nicroot on your server, mount the ThinkNIC CD-ROM on your server and copy the contents (using the -a flag) into this directory. The root directory of your ThinkNIC will be under /nfsroot/nicroot. We'll need to make some changes to this filesystem to make the system work. The kernel that comes with the ThinkNIC must use the devfs filesystem because there are no device files on the CD-ROM. We'll need to create them, so make sure you are the root user on your server. The /nfsroot/nicroot/dev directory is a symlink; remove the symlink and make a dev directory under /nfsroot/nicroot. Now use the following command to create all the device files:
cp -a /dev/* /nfsroot/nicroot/dev
We need to make one more change. When the kernel loads, it executes the commands in /etc/rc.d/rc.sysinit. This file manipulates the Flash memory and does other things that won't work because of the different kernel on our ThinkNIC. Rename that file to something else, and create a new file that has the following lines in it:
#!/bin/sh
/bin/sh
Make sure this file is executable, and it will bring up a shell when your system boots.
Install an NFS Server
Now that your filesystem is on the server, it needs to be shared with the network using NFS. On Mandrake Linux, the RPM package that needs to be loaded is called nfs-utils. You'll need to export this directory so the ThinkNIC system can see it. Create the file /etc/exports and put the following line in the file:
/nfsroot 192.168.1.0/255.255.255.0(rw,no_root_squash)
This line tells the NFS server to share the /nfsroot directory with any computer on the 192.168.1.x subnet. Replace this subnet with yours. It also tells the server to share it in read/write mode. The no_root_squash option allows root users on the remote system to access the share-with-root access on the server. This is not the default and should be used with care.
You should now start your NFS server by issuing the following command on your server:
/etc/rc.d/init.d/nfs start
Testing It Out
Go ahead and boot your ThinkNIC system. If everything is configured properly, your ThinkNIC should get an IP address from the DHCP server and load the BpBatch program that loads the kernel. The kernel should then mount the root filesystem using NFS, and you should be at a shell prompt. If that worked, it's time to celebrate. If that didn't work, check each step listed in the article. Each step in the boot process is covered, so it should be easy to discover which step didn't work and compare your server to the example here.
An Open-Source Solution
I wanted to have a completely open-source solution to this problem. Unfortunately, I couldn't make the GRUB software work, and I sure didn't want to write my own boot-loader software from scratch. The Etherboot package allows you to create your own boot ROM. If you have a network board that has a ROM socket on it, you can use the Etherboot package to create a ROM image that can then be burned into an EPROM for use on your network board. In our case, we already have a boot ROM, and it can't be replaced with a new one.
Fortunately, the Etherboot package allows you to make a ROM image that has a boot sector, so you can test the package from a floppy. This also means you can take this boot image and put it on a bootable CD-ROM. This doesn't use the boot ROM in the system so it would work on systems without boot ROMs.
You can download the source code to the Etherboot package and compile the code yourself, or you can use the ROM-o-matic web page setup by Marty Connor at http://rom-o-matic.net/. I tried both and the ROM-o-matic web page was by far the easiest. Select the SiS900 NIC and choose the Floppy Bootable ROM Image. In a few seconds, you'll have a 40K ROM image that you can use on your bootable CD-ROM.
When you make a bootable El Torito CD image, you need to provide a floppy boot image that is exactly 1.2, 1.44 or 2.88 megabytes. I used the following command to take the ROM image and append enough bytes to make it a 1.44MB file. Make sure your downloaded ROM image is in the file eb.img:
dd if=/dev/zero of=eb.img bs=512 seek=72 count=2808
This will increase the size of the file to 1.44MB. I used the following command to make the CD image:
mkisofs -b eb.img -o /tmp/cd.iso .
You can then burn to a CD-ROM the ISO9960 CD image in the file /tmp/cd.iso. Don't tell anyone, but I used Adaptec's Easy CD Creator software on Windows. If your CD writer happens to be in a Linux box, you can feel free to use one of the fine CD-writing programs for Linux.
There is a utility in the Etherboot package called mknbi. This file will take a Linux kernel and put it in the proper format needed to load via the Etherboot loader. Install this utility package, change to the /tftpboot/bpbatch directory, then run the following command:
/usr/local/bin/mknbi-linux bzImage --output bzImage.out
--param="rw root=/dev/nfs nfsroot=192.168.1.5:/nfsboot/nicroot
ip=192.168.1.12:192.168.1.5:192.168.1.1 :255.255.255.0::eth0"
This will take the kernel we generated earlier, convert it to the proper format and add all the parameters required for it to find its root filesystem.
You need to change one line in the /etc/dhcpd.conf for this new method to work. The existing file contains the line
filename "/bpbatch/bpbatch";
Change this line to
filename "/bpbatch/bzImage.out";
Restart your DHCP server so it reads your changes. Put your bootable CD-ROM into your ThinkNIC and turn it on. If all goes well, it will start up as it did using the BpBatch software. This time though, you are using a completely open-source solution.
Conclusion
It does take a little time to get all the components set up properly, but booting your ThinkNIC computer from the network is the first step to making it do new and exciting things. You did enter your exciting idea in the Embedded Linux Journal ThinkNIC contest didn't you?
About the author: Jay Sissom ([email protected]) is a systems analyst in the Computing Department at Indiana University in Bloomington, Indiana. His amateur radio callsign is KA9OKT, and he enjoys playing the piano. He wishes there were 36 hours in the day so he had more time to try out some of his crazy ideas.
Copyright © 2001 Specialized Systems Consultants, Inc. All rights reserved. Embedded Linux Journal Online is a cooperative project of Embedded Linux Journal and LinuxDevices.com.
This article was originally published on LinuxDevices.com and has been donated to the open source community by QuinStreet Inc. Please visit LinuxToday.com for up-to-date news and articles about Linux and open source.