RTLinux versus RTAI
In this document I have tried to put together the most relevant
(in my opinion) characteristics of RTLinux and RTAI.
I did my best, but I know there may be errors... this document
do not intend to be the last word on this issue.
Introduction
Two approaches to real-time Linux
There are two different approaches to provide real-time
performance with Linux:
- Improving the Linux kernel preemption.
- Adding a new software layer beneath Linux kernel with full
control of interrupts and processor key features.
There are two working groups on each approach:
TimeSys and Linux kernel
preemption project (supported by MontaVista) took the first
approach, both groups are working in parallel to produce a
similar result.
The second approach is supported by RTLinux and RTAI.
A detailed analysis of the pros and cons of the two
alternatives can be found in the paper Comparing
two approaches to real-time Linux by Tim Bird.
How Real-Time performance is achieved in both, RTLinux and
RTAI
The answer to this question can be found in the
RTAI programming Guide:
"The real-time Linux scheduler treats the Linux operating
system kernel as the idle task. Linux only executes when there
are no real time tasks to run, and the real time kernel is
inactive. The Linux task can never block interrupts or prevent
itself from being preempted. The mechanism that makes this
possible is the software emulation of interrupt control
hardware."
There are some intrinsic real-time features that are achieved
by just executing real-time tasks in kernel space:
- Real-Time tasks (threads) are executed inside kernel
memory space, which prevents threads to be swapped-out and
also also the number of TLB misses is reduced.
- Threads are executed in processor supervisor mode
(i.e. ring level 0 in i386 arch), have full access to the
underlying hardware.
- Since the RTOS and the application are
linked together in a "single" execution space, system call
mechanism is implemented by mean of a simple function call
(not using a software interrupt which produces higher
overhead).
Licenses and patents
- RTLinux:
-
RTLinux is released under two different licenses:
-
Open RTLinux
Patent License . This license allow you to use RTLinux
without fee in two situations:
- By software licensed under the GPL; or
- By software that executes within an Open RTLinux
Execution Environment - whether that software is licensed
under the GPL or not.
where "Open RTLinux Execution Environment" means: A
computer hardware system where the interrupt control
hardware of processors and system boards is under the direct
control of unmodified Open RTLinux Software in binary form.
Any modification of the code covered by the
Open RTLinux License must by GPL 2 (fully GPL 2,
i.e. the code can not be closed). It must be open and
be available on the web. Since it is NOT LGPL it is
not legal to link with non-GPL code.
-
Non-free1 License: FSM
Labs require you to buy a commercial license if you want
to impose proprietary restrictions on the use of your
products3, that is,
distribute your derived work (based on RTLinux) using a
different licensed than the GPL. The commercial license is
similar to the one used by QT
Library done by TrollTech. The main idea of these
licenses is that you can do whatever you want, but if you
make money (money change hands, between you and your
customer) then you have to buy a license.
FSM Labs has patented (U.S. Patent No. 5,995,745) the
process of capturing the interrupts and provide a
virtualized hardware to Linux so that RTLinux can take
full control of the machine to provide hard real-time
response. It is a software patent which is not applicable
to Europe, it is only valid in USA.
- RTAI:
-
Mostly LGPL 2. Which means that:
users can modify the code (Like any GPL program) but also it
is possible to use proprietary modules that can be linked or
used in conjunction with RTAI code.
The RTAI core is GPL since that work came from RTLinux
original code. The situation of the RTAI license is not
clear. It is even possible (where software patents are
enforced, no in EU) that you have to buy a license from
FSM Labs to use RTAI.
Comments
From the academic point of view, RTLinux and RTAI are the same,
you can use and modify the code as you like.
The RTAI team tries to allow proprietary development for zero
prize2 (by using LGPL
license) while RTLinux do not.
RTAI license (LGPL) seems to be not too legal, in the sense that
RTAI is based on GPL code and it is not possible to change the
license without the permission of all the code copyright
holders. In other words, derived work of GPL code can not be
released under a different license.
[Note]The use of the word "seems" in the previous
paragraph, is seen as a way to spread FUD. It is not my
intention, but I'm not a lawyer and it is difficult to give a
more detailed interpretation of the reality.
Support
- RTLinux:
-
Web:fsmlabs.com
Mail-list: Active, posted questions got answered.
Main back-company: FSMLabs (fsmlabs.com)
- RTAI:
-
Web: www.rtai.org
Mail-List: Active, posted questions got answered.
Supporting-companies: DENX Software Engineering (www.denx.de), Pengutronix (www.pengutronix.de) and
Sysgo Realtime Solutions (www.sysgo.de)
API Compatibility
- RTLinux:
- Implemented as a POSIX 1003.13 "minimal realtime operating
system". The internal design was driven by the POSIX
requirements.
- RTAI:
-
RTAI supports it's own API derived from the RTLinux V1
API. The new features (message queues, mailboxes, etc) do not
follow a coherent API and are incompatible between them. The
same feature is implemented in several ways with different
systems calls.
RTAI maintains compatibility with the V1 RTLinux API.
There is a POSIX module which provides POSIX 1003.1c.
(PThreads) and 1003.1b (Pqueues).
Architectures
- RTLinux:
-
i386, PPC, ARM.
- RTAI:
-
i386, MIPS, PPC, ARM, m68k-nommu.
Development Status
- RTLinux:
-
Development stalled at version "3.1" (May-2001).
There are known bugs.
Linux kernel supported: 2.2.19, 2.4.4 (unofficially 2.4.18, no SMP support).
- RTAI:
-
Actively developed.
Linux kernel supported: 2.4.18
Comments:
Current Linux kernel is 2.4.18 (06-03-2003).
Debug
- RTLinux:
-
- Source-level debugging using GDB (DDD), with SMP support.
From within the same target machine (no cross debugging required)
- Tracer. It can trace kernel and application events.
- POSIX trace.
- RTAI:
-
- kgdb:
Source-level debugging from a host linked by a serial
line.
- Linux Trace Toolkit (LTT). It
is a full-featured tracing system for the Linux
kernel. It includes both the kernel components required
for tracing and the user-level tools required to view the
traces.
Memory management
- RTLinux:
-
- Dynamic memory
-
By default, memory must be reserved before the real time
thread has started. No dynamic memory allocation
(malloc and free functions) are available.
- Shared memory (RTLinux<->Linux)
- MBUFF module (mbuff_alloc,
mbuff_free) (Non-POSIX)
- RTAI:
-
- Dynamic memory
-
Support for dynamic memory allocation
(rt_malloc, rt_free). This functions
do not have a hard real time behavior. (Non-POSIX)
- Shared memory (RTAI<->Linux)
-
- MBUFF module (mbuff_alloc, mbuff_free) (Non-POSIX)
- SHMEM module (rtai_malloc, rtai_free,
rtai_kmalloc, rtai_kfree). Shmem is the
RTAI version, developed by Paolo Mantagazza, which
operates in much the same way but is dependent upon RTAI (Non-POSIX)
Interprocess communications
- RTLinux:
-
- FIFO
-
UNIX PIPE's like communication mechanism that can be
used to communicate real-time process between them and
also with normal Linux user processes. (non-POSIX)
- Mailboxes
-
Jerry Epplin IPC implementation (rt_mq_open, rt_mq_send, etc. ) (obsoleted) (Non-POSIX).
- RTAI:
-
- FIFO
-
UNIX PIPE's like communication mechanism. Same than in
RTLinux. (non-POSIX)
Also provides a mechanism to create fifos by name.
- Mailboxes
-
RTAI provides mailboxes. Messages are ordered in FIFO
order. Different size messages of are allowed. Multiple
senders and receivers can read and write messages to
the same mailbox. There are several sending an
receiving functions that provides a lot of flexibility:
blocking, non-blocking, timed and whole/partial
message.
- Message queues
- Provides four different (incompatible) intertask messages facilities
- Message queues compatible with POSIX 1003.b
queues. The functionality is provided by the POSIX
compatibility module.
- Small (4 bytes) synchronous messages. Messages can
be priority ordered (determined by the compile time
option MSG_PRIORD). Also there are timed sending and
receiving primitives.
- Extended intertask messaging. Allows to use
intertask messages of any size.
- Remote Procedure Calls (RPCs) do the same thing that
synchronous messages but the tasks are coupled awaiting a
reply from the receiver. RPCs operate like complementary, send
and receive message pairs. (Non-POSIX).
- net_rpc
-
RTAI has extended its API to allow remote (other
host) procedure call RPC. New API functions has been added with the
following syntax: replace the first two letters of the function name
(for example: given the rt_mbx_send(), the new
function RT_mbx_send() has been added); and the
new function has two new parameters: node and port. This
feature do not comply with any communication standard.
Comments:
Too many incompatible, redundant and non standard features is
not desirable. A RTOS should provide a simple API.
Synchronization
- RTLinux:
-
- Mutex
-
POSIX thread mutex variables. Provides PRIORITY_PROTECT
protocol to deal with the priority inversion problem.
- Condition variables
-
POSIX conditional variables.
- Semaphores
-
POSIX semaphores.
- RTAI:
-
- Mutex
-
POSIX thread mutex variables. Provides PRIORITY_INHERIT
protocol to cope with the priority inversion problem.
- Conditional variables
-
POSIX conditional variables.
- Semaphores
-
- POSIX semaphores.
- RTAI semaphores (rtf_sem_init,
rtf_sem_wait, ...). The underling mechanism
used to implement this semaphores is the FIFO's
blocking features.
User-Space Real-Time
- RTLinux:
-
User space real-time signals. Hardware interrupts and timed
signals can be handled at user space signal handlers. It is
not possible to call any Linux system calls nor RTLinux
services from these handlers.
New features in the commercial distribution.
- RTAI:
-
Well developed and tested. Caller LXRT. Same
API than in kernel-space real-time, except the POSIX threads
API.
There are three different implementations:
-
LXRT services (Soft-hard real time in user space)
-
Soft-hard real time in user space API
|
rt_task_init rt_sem_init
rt_mbx_init rt_get_adr
rt_register rt_get_name
rt_drg_on_adr rt_drg_on_name
rt_allow_nonroot_hrt
|
LXRT is a module that allows to use RTAI services in user
space.
It is implemented by using RTAI process (being executed
in the kernel space) for every LXRT process to perform
the actual function call. Every time a LXRT process
calls a RTAI API function, the parameters are copied to
the RTAI peer process which is the one that to the real
job.
It is possible to use any Linux System calls from a LXRT
process, as well RTAI services.
LXRT user space tasks can interact with kernel
space RTAI tasks by using the same API (share memory, send
messages, use semaphores, etc.).
-
Extended LXRT (Hard-hard real time in user space)
-
Extended hard-hard user space API
|
rt_make hard_real_time
rt_make_soft_real_time
|
This is an extension to the original LXRT facility. By
calling the function rt_make hard_real_time(), a
LXRT process gets even higher "priority". The process is
scheduled by the Linux kernel but from a
bottom-half (a bottom-half is an internal Linux kernel
utility user to run interrupt services).
From: RTAI Programming Guide 1.0, page 139 (See references):
"The call to
rt_make_hard_real_time allows to take a normal process
out from the Linux running queue by calling schedule()
after having queued the task to a bottom handler. When
the bh runs, the task is scheduled as a hard real time
module by lxrt_schedule(), and Linux will not know of
it, while having it still in its process list."
These kind of processes should not call any Linux System
call that can lead to a task switch, otherwise a
deadlock may occur.
-
Mini RTAI LXRT
-
Mini RTAI functions
|
rt_task_init rt_task_delete
rt_insert_tasklet rt_remove_tasklet
rt_tasklet_exec
rt_timer_init rt_remove_timer
rt_set_timer_priority rt_set_timer_period
rt_set_timer_handler rt_set_timer_data
|
This module provides a way to execute functions in a
similar way the Linux Kernel 2.4 tasklets are executed.
These taskslets are of two kinds: normal and timed (timer).
They can be used to implement Programmable Logic
Controllers (PLC), and other simple real time
sub-systems.
Blocking system calls (Linux or RTAI) should not be called
from tasklets.
Currently, (RTAI 24.1.9) Mini_RTAI_LXRT and LXRT
processes can not be jointly used.
Comments
There are two main reasons to run real-time applications in user space:
- A task crash due to a programming bug do not affect the
system stability. The crashing task is terminated, but the
system remains safe. Which yields to faster development.
- It is easier than inserting modules into the kernel. The
real-time application is coded and started-up like a normal UNIX
process.
By using POSIX API, it is possible to use the same API both in
user and kernel space. Therefore, it is worthless to "port"
(bring RTAI or RTLinux API) to user space process. In fact,
using Xavier Leroy PThreads [Xavi] it is
possible to implement soft RT applications in user space. Also,
the new preemptible kernel reduces system latency, so it is
possible to have soft real time in normal Linux user processed.
By using user space processes all the advantages of running
inside the kernel space are lost: more TLB misses and heavy
system call mechanism.
In a recent discussion in the Linux-kernel mailing list, Linus
Torvalds talked about user space real-time (see marc.theaimsgroup.com)
...
With RTLinux, you have to split the app up into the "hard
realtime" part (which ends up being in kernel space) and the
"rest".
Which is, in my opinion, the only sane way to handle hard
realtime. No confusion about priority inversions, no
crap. Clear borders between what is "has to happen _now_"
and "this can do with the regular soft realtime".
...
Documentation
- RTLinux:
-
- RTAI:
-
Although RTAI documentation is well written, it is not
maintained. New features are not well documented while
deprecated and not supported functionality is still in the
documentation.
Miscelanea
- RTLinux:
-
- RT-Lab: Real-Time Linux Experiment Interface System.
- RTAI:
-
- Octave, MatLab &
Simulink interface: generate code for RTAI from
Simulink Workshop.
- Watchdog module: the highest priority periodic task that takes
protects the systems against tasks errors. Available policies:
- Nothing.
- Resynchronize the task's frame.
- Debug.
- Stretch the period of the offending task.
- Slip the offending task by forcibly suspending it for
a percentage of its period.
- Suspend the offending task.
- Kill the offending task and remove all trace of it.
- System information through proc interface.
- Scripting language bindings: PERL.
- Xenomai:
a project is to help creating emulators of traditional RTOS
APIs that ease the migration from these systems to a
GNU/Linux-based real-time environment. The following API's
are available
- pSOS+ emulator (complete)
- VRTXsa emulator (complete)
- VxWorks emulator (complete)
- uITRON implementation (underway)
- Both:
-
RT-NET: UDP-IP.
Comedi: linux
control and measurement device interface.
API functions
The list of system calls is not included in this document, but
in a separate file: API.
Summary
The following table summarizes the differences. Features that
are almost the same in both systems are not included:
� |
RTLinux |
RTAI |
Company |
FSM Labs |
DENX Software Engineering, Pengutronix and Sysgo Realtime Solutions |
Arch |
i386, PPC, ARM |
i386, MIPS, PPC, ARM, m68k-nommu |
License |
GPL - Commercial |
LGPL - GPL |
API |
POSIX |
custom |
Memory |
shared |
dynamic and shared |
Debug |
GDB, Event tracer |
Cross GDB, LTT |
IPC |
FIFO's |
FIFO's, Named FIFO's, Mailboxes, messages, PQueues, net_rpc |
User space Real Time |
Signal handlers |
LXRT soft, LXRT hard, Mini LXRT |
Misc |
RT-Lab |
Octave, proc, Watchdog, Scripting, Xenomai |
References
-
Comparing real-time Linux alternatives
- Comparing two approaches to real-time Linux
- The RTAI Manual
- RTAI Programming Guide 1.0
- Tutorial del API de RTLinux (Spanish)
- Xavier Leroy implementation of POSIX
threads.
- Real-time Linux sub-kernels, benchmarks... discussion
Notes
- (1) non-free
- The term "non-free" is preferred to "commercial" in this
sentence (See FSF recommendations Commercial)
- (2) for zero prize
- The term "for zero prize" is preferred to "for free" in this
sentence (See FSF recommendations For Free)
- (3) sell
- The term "imposing proprietary restrictions" is preferred to
"sell" in this sentence (See FSF recommendations Sell Software)
This research was supported by the IST 2001-35102 OCERA project.
Please note: This document has been reproduced by LinuxDevices.com with permission of the author.