News Archive (1999-2012) | 2013-current at LinuxGizmos | Current Tech News Portal |    About   

ELJonline: Programming for EyeTap Systems

Nov 1, 2001 — by LinuxDevices Staff — from the LinuxDevices Archive — views

An “EyeTap” is a camera/processor/display device you can wear. Here, two real-life cyborgs play by reversing X displays.

Over the past few years, we have been working on a number of programming environments for EyeTap systems. A common feature of EyeTap systems is the diverter, which renders image content as left-right reversed or top-bottom reversed. Accordingly, we have been developing a number of options and methodologies for dealing with the problem of reversed display.

Although early cathode-ray-tube EyeTap systems relied on swapping deflection yoke wires (in color EyeTaps one must be careful to swap only the horizontal yoke wiring, otherwise images are chopped off because of the critical timing of the filter wheel as synchronized with the vertical deflection), certain conditions arise where it is inconvenient to reverse the hardware.

Moreover, because the sensor apparatus (whether it be a camera or similar sensory system) also sees the world in reverse, it is often desired that the display actually portray a backward image to compensate properly, and thus the two will reverse each other and cancel out. Without reversing X, images would appear to the viewer as shown in Figure 1.

figure

Figure 1. An Image without Reversing X

One early attempt was a program called caplive, developed by Mann and extended by Mann, Fung and several others into something called xcaplive, along with a set of utilities called glinaccess, meant for use with EyeTap systems.

More recently, a thesis project proposed by Mann and carried out by Manders involved the reversal of XFree86 to facilitate use of XFree86 with an EyeTap system.

With the restructuring of the XFree86 code of late, XFree86 4.0.x has provided an easy means of left-right reversing the Xserver. Specifically, new installations of XFree86 version 4.0.x have an option to make use of a shadow buffer. In a regular install of XFree86 4.0, most chipsets will offer the option of rotating the Xserver by pi/4 radians. This is due to the efforts of Mark Vojkovich.

The initial code provided in the XFree86 4.0 distribution performs a harder task than what is needed to left-right reverse the Xserver. Any one of the drivers provided in XFree86 4.0 will have essentially the same structure that will be discussed here later on, but for the purpose of example, the NVIDIA driver (nv) has been chosen. Note that this manipulation has been performed successfully on almost all of the XFree86 4.0 drivers.

First, it is advantageous to examine the code to which the necessary modifications will occur. After expanding the XFree86 4.0 source code (available at ftp://ftp.xfree86.org/), we get the file …/xc/programs/Xserver/hw/xfree86/drivers/nv/nv_shadow.c. Listing 1 comes from this file.

Listing 1. Code from nv_shadow.c

Now, this code is active when the Shadow Buffer option is selected in the XFree86Config file. All it is doing is writing from allocated memory to the graphics card. Understanding this code is not that difficult if we consider what some of the variables are.

The sections of the hardware memory-mapped regions (pointed to by pNv->FbStart) that need to be redrawn because of changes are outlined by pboxes. A pbox is simply a set of four offsets, two x offsets and two y offsets, and num is the number of pboxes that need to be redrawn. The width and height of the pboxes may be calculated from these offsets.

The code in Listing 1 redraws on pbox at some time upon each iteration of the outer while loop, and src and dst are the memory addresses of the source and destination addresses in memory. The real changes that need to be made are that we must write the rgb information to the destination maintaining the order of the rgb bits, but one level of abstraction higher. These bit groups must be written in reverse order. This means we must write the pixels out in the destination starting from the pbox x coordinate opposite to the code in Listing 1. This may be accomplished by changing the following:


 dst = pNv->FbStart + (pbox->y1 * FBPitch) + (pbox->x1 * Bpp);

to


 dst = pNv->FbStart + (pbox->y1 * FBPitch) + ((pScrn->virtualX - pbox->x1) * Bpp)

This will get us to the other end of the pbox in the representation in memory.

At this point, the code is beginning to read the source address from the correct location. For each horizontal line, the actual groups of rgb bits must be written out in reverse. This implies that the inner while must change as well.

Unfortunately, we can't write the bits out one “width” at a time (which is what the memcpy is doing in the original code). We must traverse the sections in finer increments, which leads us to change


 while(height -- ) {
        memcpy(dst,src,width);
        dst += FBPitch;
        src += pNv->ShadowPitch;
    }

to


 while(height -- ) {
       for (i= 0; i          memcpy(dst-i, src+i, Bpp);
       dst += FBPitch;
       src += pNv->ShadowPitch;
    }

The previous code takes care of the painting of the pixels. Next we need to correct the positioning of the mouse (which is not yet reversed). A subrountine at the start of nv_shadow.c, named NVPointerMoved, allows us to do this. If we look at the NVPointerMoved subroutine, the code is used to change x coordinates to y coordinates and similarly to change y to x. This is, of course, not needed to reverse the Xserver, but the code may be modified to perform the function needed. Listing 2 has been commented to outline the necessary modifications.

Listing 2. Modifications to the NVPointerMoved Subroutine

What has been done in the new section of code is to take the original y coordinate and leave it unchanged and subtract the original x coordinate from width of the screen. This is all that is needed for left-right reversing the mouse. The variable pNv->Flip is important and will be explained in what follows.

To highlight further all the changes that have been made to this point, a completely modified example of nv_shadow.c is available [see Listing 3 at ftp://ftp.ssc.com/pub/elj/listings/issue06/4779.tgz].

The changes so far will left-right reverse the Xserver; now all that is needed is to get the driver to call it. Certainly the easiest way is to add an option to the XFree86Config file such that when called, the Xserver would be left-right reversed.

We need to get our new option (which we will call FlipLR) parsed by the driver. Specifically, when we add option “FlipLR” to the device section of the XFree86Config file we want to have our Xserver reversed.

Taking a careful look at nv_driver.c around line 299 [see Listing 4 at ftp://ftp.ssc.com/pub/elj/listings/issue06/4779.tgz], we find the structure shown in Listing 5.

Listing 5. Code from nv_driver.c

These are the usual options available to the driver. They will differ slightly from driver to driver, but some similar structure exists in each of the drivers. What is needed in the structure is a line that should appear as:


 /* our new cyborg enhanced option */
 { OPTION_FLIP,              "FlipLR", OPTV_BOOLEAN,   {0}, FALSE },

The subroutine NVPreInit is used to parse the XFree86Config file and set various structure parameters to the correct values. The lines that need to be added can occur within a wide range of the subroutine. In keeping with the overall design, with the NVIDIA driver, it makes sense to add some code at line 1061 (right after the Rotate option is parsed):


 if ((s = xf86GetOptValString(NVOptions, OPTION_FLIP))){
    pNv->ShadowFB = TRUE;
    pNv->NoAccel = TRUE;
    pNv->HWCursor = FALSE;
    pNv->Flip = TRUE;
    xf86DrvMsg(pScrn->scrnIndex, X_CONFIG,
        "Left Right Reversing the screen - acceleration
         disabledn");
 }

This will set the required variables in the pNv structure to the values necessary for left-right reversing the Xserver. At this point, the field pNv->Flip doesn't exist. This needs to be added to the header file and will be dealt with later.

If the variable pNv->Flip did in fact exist, we would have parsed the XFree86Config file properly. Now what is necessary is to have the function NVRefreshAreaFlip used when needed. This is done in NVScreenInit. Within NVScreenInit, there is a section dealing with the shadow buffer. Using a reasonable editor to make the changes, you can search for the string “if(pNv->ShadowFB)” in the NVIDIA driver; this occurs around line 1,779 of nv_driver.c. The original piece of code is shown in Listing 6.

Listing 6. Code from nv_driver.c

Since we want to add in the left-right reverse functionality, we can modify Listing 6 to consider our reverse X cyborg option by changing it to what's shown in Listing 7.

Listing 7. Adding Left-Right Reverse Functionality to NVScreenInit

All that is left to do is add the appropriate entries to the header file. The name of this file varies depending on the driver. Of course it is always a .h file, and that narrows the field greatly. All drivers will have a similar header, but for the NVIDIA driver, the file is nv_type.h. If the modifications are on a different driver, it's probably best to look at the NVIDIA .h file and grep for some similar entries in the directory of the driver that is being modified.

The pNv->Flip field must be defined. There is a struct that will be similar to Listing 8. To this listing we simply want to add the entry


 Bool              Flip;

Listing 8. Defining the pNv->Flip Field

Lastly, we want to add our newly created function for left-right reversing to the header. This probably belongs here


 void NVRefreshArea(ScrnInfoPtr pScrn, int num, BoxPtr pbox);

and should be declared as


 void NVRefreshAreaFlip(ScrnInfoPtr pScrn, int num, BoxPtr pbox);

Now, we may safely reverse the Xserver by adding the line Option "FlipLR" to the XFree86 4.0 XFree86Config file, which should be located in the directory /etc/X11.

Just in case any of the modifications above were not clear, a modified version of the nv_driver.c is available [see Listing 4 at ftp://ftp.ssc.com/pub/elj/listings/issue06/4779.tgz], and the header file nv_type.h also is available [see Listing 9 at ftp://ftp.ssc.com/pub/elj/listings/issue06/4779.tgz].


About the authors:

Dr. Steve Mann is regarded by many as the inventor of the wearable computer, of the EyeTap camera (a device that causes the eye itself to function as if it were a camera) and of the Reality Mediator (wearable image processor for the visually challenged, http://wearcam.org/mr.htm). He is currently a faculty member at the University of Toronto, Department of Electrical and Computer Engineering.

Corey Manders currently holds degrees in Computer Science (University of Toronto) and Fine Arts (York University) and will begin a Master's Degree in Applied Science at the University of Toronto starting in September 2001 under the supervision of Professor Steve Mann. For the last year he has been a resident cyborg in Mann's Humanistic Intelligence Lab.


Copyright © 2001 Specialized Systems Consultants, Inc. All rights reserved. Embedded Linux Journal Online is a cooperative project of Embedded Linux Journal and LinuxDevices.com.

 
This article was originally published on LinuxDevices.com and has been donated to the open source community by QuinStreet Inc. Please visit LinuxToday.com for up-to-date news and articles about Linux and open source.

Related Posts:



Comments are closed.