ELJonline: From Pockets to Packets
Jan 1, 2001 — by LinuxDevices Staff — from the LinuxDevices Archive — 1 viewsHow PocketLinux and Jabber's XML instant messaging open a whole new development platform.
In September (Issue 77), our cover story was titled “The Next Bang: The Explosive Combination of Embedded Linux, XML and Instant Messaging”. It was one of those pieces that was almost entirely about enabling technologies, rather than the goods they produced: a watch this space kind of thing. At the time there was plenty to find in each of three named categories but nothing yet in that shared space.
Now there is. The first thing to appear in the space (according to the parties involved–we're in open-source territory, so there may be others) is what's coming out of work going on between the “Jabber Guys” (both at Jabber.org and Jabber.com) and Transvirtual, the small but inveterate embedded development company that conspicuously offers PocketLinux, which they demo'd amidst eager mobs of curious programmers, at both LinuxWorld Expo in August and Atlanta Linux Showcase in October.
In November at Comdex they will be demonstrating the first results of this new collaboration: two Compaq iPAQ handhelds exchanging XML streams over Jabber's all-XML open-source instant-messaging system.
Yesterday I talked with three of the Transvirtual guys: Timothy Wilkinson, Founder, President and CEO; Tony Fader, Vice President of Marketing and Paul Fisher, Senior Developer. I successfully recorded the conversation with Tony and Paul, and the results follow.
ELJ: Are you the first to put embedded Linux and XML-based instant messaging together in some way?
Paul: Yes. At least we're the first to use Jabber, which is XML-based instant messaging, as a generic transport.
Tony: There are several leading-edge things Transvirtual is doing that are attracting quite a bit of interest. First, we've announced the opening of the Custom Edition of Kaffe, our clean-room implementation of a Java-compatible Virtual Machine. The Custom Edition of Kaffe has extensions and optimizations that are targeted directly at the embedded space. Second, we've built a framework down into Linux and up into XML that allows you to write your applications in XML and Java and then run them on Linux. Third, as Paul pointed out, we're using Jabber as our transport system.
Paul: There is actually a lot of application work that can just be done in XML-land, which is kind of cool.
Tony: Most applications today are moving toward a document-centric, web-centric model, so XML is obviously an exceptionally elegant solution.
ELJ: Tell us more about Kaffe. It's a JVM–
Tony:It's Java-compatible, but for legal reasons, we can't say it's a Java Virtual Machine. The fact of the matter is that it's no longer “just a JavaVM” in any case, so we prefer to call it Kaffe OpenVM.
ELJ: It's yours, right?
Tony: Kaffe is the community's. The story behind the birth of Kaffe is that Tim Wilkinson read the license for Java 1.0 upon its release and said, “Hogwash, I'm not signing that.” He then proceeded to clean room the initial specification in his spare time over the course of next three weeks. The first implementation of Kaffe was made available for FreeBSD in late 1996.
ELJ: And how is it more than Java?
Tony: It's really a superset of the Java specification. We have extensions to our AWT (Abstract Windowing Toolkit) to support the integrated framebuffer graphics library. We have extensions to Kaffe for integrating XML. We have extensions down into Linux for implementing things like Video for Linux and MP3 playback.
ELJ: BSD was in the embedded space then?
Tony: No, that first BSD version was for the Desktop Edition (GPL). At that time, we maintained a custom edition in-house as well. To generate revenue we ported the custom edition to embedded OSes like Wind River's VXworks, LynxOS, SMX, WinCE and others. At one point we had ports of Kaffe to something like 20 different embedded operating systems. Of course, all those operating systems are now being overwhelmed by the tidal wave that is Linux. Today, because of Linux's exceptional portability, scalability and open-source, we see the OS market for the embedded space in a more liquid state than at anytime in the last 20 years.
ELJ: In the early 90s I hung around the embedded space when I worked with Hitachi launching the SH processor. In those days the embedded world was in the Middle Ages. Every industry was an isolated city-state that rose like a stovepipe from the arcane combination of some processor and some OS.
Tony: You rolled your own, yes.
ELJ: Each profession was isolated, separate and noncommunicative. Security systems, factory automation, process control, you name it–they were all off in a world unto themselves. You can even see that in your own home when you hire professionals to install intelligent telephone, security, computer networking and home entertainment systems. They're all entirely separate professions, using entirely separate solutions. But just this year when we started covering the topic in depth, we found that Linux had kind of taken over, and there is suddenly the promise of embedded intelligence connecting into the Net and the Web. Suddenly you can develop applications that work across and through all these different professions. It's all hugely promising, but where would you say we stand right now?
Tony: I would say that Linux has just emerged as a viable solution in the embedded space. Perhaps just in the last year. What we saw in classical embedded systems was motion towards POSIX compliance. We saw companies–Tandem, Wang and all these other old terminal systems, each with their own standards and problems. With the advent of real networking and open-source developmental tools, we saw motion in the entire embedded space toward POSIX compliance. It was a very small and logical step to go from closed source POSIX-compliant operating systems to open-source POSIX-compliant operating systems. And that's what Linux is.
Paul: And we're getting chips fast enough now to run Linux. And memory is a lot cheaper these days, along with processing power.
ELJ: It's interesting to me that you guys have been around embedded for all this time. One interesting thing is that a bunch of these classical embedded OS guys are now vending Linux. Colin Hunter is a founder of Transmeta. Jim Ready has MontaVista. Michael Tiemann and Cygnus are part of Red Hat. Lynx is now LynuxWorks.
Tony: Yes, momentum in the industry has been amazing. It's really very exciting.
ELJ: It's amazing to me that while everybody is waiting for Linux to make it on the desktop, it seems to have leapfrogged down into embedded devices all over the place. It may still be nowhere–relative to Windows–on the conventional desktop and highly competitive in servers, but it is really, at the very least, in a position to win in the embedded space.
Paul: And it's not like Linux is ever going to go away, either.
Tony: When I moved from development into marketing, I went back to Windows just so I could interoperate with the marketing teams at outside companies. Now, I'm in the process of moving back to Linux. The simple fact is that Windows just can't keep up anymore. Linux development is moving way too fast. Because of this real-world experience, I would argue that Linux is starting to become viable even on the desktop.
ELJ: The ironies abound. It's so easy to get into the versus thing. Linux is supposedly engaged in these battles for desktops and servers. But if you look at the numbers, both Linux and Windows are succeeding in both spaces. Both are growing. Nobody is losing here. But in the embedded space, Linux is on a huge roll. I think Linux really does look like the big winner here. It's kind of a baseline thing. A fundamental commodity rather than a competitive OS in the usual sense.
Tony: Your point about Linux leapfrogging the classic desktop and going into these small resource-constrained environments is absolutely true. It is how things will be.
ELJ: You guys came up with PocketLinux, which has been a huge hit at the Linux shows. I couldn't even break through the crowd to talk to you.
Tony: Yeah, it was madness. We love it!
ELJ: You had the VTech Helio there, which I gather is a development platform, primarily. Where do you guys see that business going? I'm talking about the non-Palm PDAs: the Helio, the Casio. They're more muscular than the Palms, but they have some drawbacks too–size, for example.
Paul: The Palm, unfortunately, is still too weak in the processor to support Linux in a useful way. It's really made just to do what it does.
ELJ: You need these other, somewhat more muscular PDAs, to run Linux, then.
Paul: Yeah. And there are trade-offs.
Tony: They eat through batteries. They're just atrocious at that. Environmental accountability is a very real concern for us. Presently the Helio runs off AAA batteries, which certainly leaves something to be desired.
Paul: We just put power management into the Helio.
Tony: And we've recommended to VTech that they go to a rechargeable battery model, which they are doing, I believe.
ELJ: Still, everybody's trying to keep down–rather than up, size-wise–with the Palm, which kind of invented the form factor.
Tony: Right.
ELJ: But the key thing at this stage, it seems, is that you now have a Linux development platform that is very inviting to third parties. Add in XML-based instant messaging, and it opens up all kinds of possibilities.
Tony: It's not just XML messaging. The GUI and data layers are XML too. XML forms the entire presentation layer of the device. The GUI can be written like web pages, and applications are written like web apps. It's an elegant solution to some very challenging problems.
Paul: In the case of PocketLinux, it's really just XML being shipped around the whole system internally. So it makes sense to ship it around externally as well, which is where Jabber comes in. We're both all-XML.
ELJ: And if you look at XML as something you ship–that you transport–Jabber starts to make sense.
Paul: Exactly, it provides an addressing and delivery mechanism for XML data.
ELJ: XML routing, basically.
Paul: Right. If we have an XML document, we just wrap it up in a Jabber packet, address it, ship it off to the Jabber server and that's all we care about. It's the server's responsibility to make sure it gets delivered.
ELJ: A handy thing with Jabber is that, unlike the closed-server instant-messaging systems (AOL and ICQ especially), Jabber deploys, like Apache or Sendmail, as a freely available server-based Internet service.
Paul: Yes. And the Jabber server is also designed to be very modular, so it's easy to plug other services into the server.
ELJ: How big can it scale? Can Jabber serve a zillion clients at one time?
Paul: There is a rewrite of the Jabber server right now that allows up to 50,000 users on one server. Since Jabber is highly distributed, it's easy to have a server farm that allows scaling to an infinite amount. Just like the web scales. And it's really fast. They've been doing some very good work at scaling it up.
Tony: My understanding is that they have rewritten it to make it extraordinarily scaleable, which is really optimal for our use.
Paul: On the commercial side they're angling towards enterprises, telephone companies and ISPs, where you just have an insane number of users.
ELJ: One of the things I want to explore with you guys is what Jabber and embedded XML-based IM do that is inherently extremely different than what one gets from conventional IM, which we associate with buddy lists and chat and AOL and ICQ. Conceptually, passing around XML documents is portentous for all kinds of interesting things that have little or nothing to do with those traditional IM activities. How are you guys looking at that, and how are you imagining what people are going to be doing with it? It seems to me that we have to work a bit at breaking away from the legacy meanings of IM.
Paul: One of the reasons that the Jabber.com guys are so into us is that we're both interested in real-time conferencing–anything that's relatively real-time. One of the things they recently rolled out was the RSS news feature, where say, using your client you could sign up to get a certain news feed. So we have this Weather Underground example. Whenever the weather changes, an instant message is sent out as a fresh news item. You can have weather updates pushed out over XML instant messages from Weather Underground to clients who need local weather reports, on say, their PDA.
ELJ: I see a really good business in being a developer here.
Tony: Especially since it's really easy to throw together an application–so easy it's like writing active web pages.
Paul: That's what we're so excited about. Throw together some XML, a couple lines of Java and you're done. It's a lot more rapid than developing, say, standard Java applets or programs.
ELJ: Give me some examples, even if they are not developed yet, of what this invites.
Paul: You need your standard PDA applications, which have to get written.
ELJ: Meaning that somebody has to replicate what the Palm does.
Paul: The idea, however, is to make the apps network-aware. We're moving to a world where you're constantly connected. That's a good thing. If people need to reach you, they can. If you need to reach somebody else, you can. Applications will presume that you're already connected to the Internet. So they can always grab or send the data they need.
ELJ: I look at the VTech Helio, and I see something that still doesn't do what the Palm does but is aware of the Net, which the Palm is not.
Paul: That's pushing it a bit with the Helio. It's really the next generation of PDAs that are going to be connected.
Tony: The Helio is a great development because of its limited resources. It is a challenge to write effective, functional Palm III-style applications that are ready to connect to the network in such a constrained environment. Something to point out about PocketLinux is that we've extrapolated away from hot-syncing the Helio to a desktop machine and instead are syncing directly to the web. This is something that we're building into the framework from the ground up.
Paul: So, say you enter in a phone number, or some sort of contact on your PDA. You sync onto your Web site and that can go to your telephone, for instance. There is no reason you should enter anything more than once. The data is stored on a Jabber server, which can just be picked up by whatever device needs that data.
ELJ: Within six feet of me are two Linux boxes, two Macs, two PCs and two Palms. If what I'm syncing fundamentally is XML data that lives on a Jabber server, I can sync all of those. Right now getting those two Palms to deal rationally with more than any one of those boxes at a time is nearly impossible. I'm waiting for a shared calendar. What you guys are doing gives me hope–something web native that ignores the hardware platform crap seems the way to go.
Paul: The two areas we're focusing on right now are PDAs and the Web. The idea is to store your calendar–or whatever–on a Jabber server as an XML document.
Tony: There are open-source projects, like the Gnome e-mail system, that use XML as the data storage format. This makes syncing a simple thing.
Paul: Since they're based on Gnome, all of Helix Code's formats are XML. We're actually working on a calendar program right now. When it's done, it'll be able to grok their format. You'd store it on a Jabber server, and whether you want to grab it with your Helix Code Evolution app, from the Web, or from your PDA, it's easy to do.You might also want to check out Brainfood, which does a lot of our Web development for us. They're involved with Debian.
ELJ: And coordinating changes would be up to the application observing all this?
Paul: You'd essentially just issue an XML request that says, “Please get me Doc Searls' current calendar.” Then the Jabber server would respond back with this XML that's your current calendar. It would be the responsibility of the calendar app to figure out what to do with that data. That's really a good example of using Jabber as a generic XML transport.
ELJ: So the app developers are essentially writing to that transport as a platform?
Tony: Right.
ELJ: So how did you discover Jabber? What happened there?
Tony: The Jabber guys kind of tracked us down at LinuxWorld, and requested that we investigate working together. We were swamped, so we didn't get around to looking at their technology until later. When we did, we found it to be a perfect match. Their technology really meshes well with what we're doing. There are other messaging systems, but they're not XML-based, or they aren't designed for instant messaging per se.
ELJ: Does what you're doing, plus what the Jabber guys are doing, plus the state of the art with the Linux kernel, plus trending in mobile devices like phones and PDAs…do these add up to a platform that flat-out invites in a whole new category of development? Are we at the point where I could put those together, plus a development plan, go to a VC and get funding?
Tony: To write applications?
ELJ: Yes.
Paul: I'd have to say yes.
Tony: Yeah. I guess I'd have to agree. Just look at the proliferation of open-source solutions in our industry. If you've been watching the growth of Linux in general, and the blossoming of Linux in the embedded markets in particular, it's already very compelling. When you add to that equation the elegance of Kaffe OpenVM and XML as a solution to a complex set of challenges having to do with hardware differentiation…well, PocketLinux gets very, very attractive.
Paul: We actually have a ship date coming up as well, going out to various vendors, in the Comdex time frame. This is what people will be writing applications for. It's the PocketLinux framework.
Tony: If you look at these systems and understand the ubiquity of these standards, it becomes clear that this is an optimal set of tools. I would like to point out that not one medical app has been written for this framework yet, not one logistics app, not one shipping, manufacturing or warehousing app. The developmental paradigm here is to be able to write an application that can be fitted to any number of different sectors using this license-free framework and this framework allows for the rapid development of cross-platform applications. It's extraordinarily compelling.
Paul: The current demo applications we have for showing off the application framework are proof-of-concept things. So it's really a case of having a framework that's ready for developers coming in–including ourselves.
ELJ: I would imagine that this makes you ripe for acquisition.
Tony: I'll tell you, Doc, I'm having a blast. I'm having so much fun. When we founded this company in 1997, it was Tim and myself, with Peter [Mehlitz, now Vice President of Engineering], still in Germany. Tim and I used to work in an unheated room down on the fourth Street promenade in Berkeley. Today, to see the cream of the crop of Linux, Java, XML and embedded technologists working feverishly on this project here, is exceptionally gratifying. I don't want to stop doing this. Acquisition isn't on my radar. I want to bring these tools to the industry and the community.
ELJ: Would you call PocketLinux an embedded distribution, then?
Tony: It's an end-to-end platform. A complete solution. PocketLinux as a system is comprehensive in its scope.
ELJ: This works on any of the various processors? StrongArm, PowerPC…
Paul: That's where Linux benefit comes in. We don't have to spend our time writing the core operating system.
ELJ: Help me out a little bit with this. Linux originally addressed an x86 instruction set. So how do you deal with the differing metal there?
Paul: If you can give us a device that runs Linux, we can port PocketLinux to it.
Tony: I believe Linux has been ported to more processors than any other operating system, by an order of magnitude. That's one of the reasons why it was so attractive to us.
Paul: In the case of the MIPS port for the Helio, we're doing a lot of the kernel work there for power management, in terms of speaker support and other areas like that. Lots of other developers are out there too, making these kernels better and faster and porting them to different hardware.
ELJ: Does any of your work end up back in the main kernel?
Tony: All the time.
Paul: There are a couple of little branches of the kernel for some of the embedded ports, and our stuff has made it into those. What's used for the MIPS, for example, is back in that tree.
Tony: We do not maintain an internal CVS (Concurrent Versions System) tree for the kernel. We have people who hack the kernel, but those changes and optimizations are handed directly back into the Open-Source community and freely available via CVS trees on the Web.
ELJ: What happens two or three generations from now, when these devices get mobile and connect via Bluetooth?
Paul: That's when you start to realize the benefits of the platform. The killer app is wireless.
ELJ: What are the gating factors for that? Is it Bluetooth? Is it guys putting this in hardware? Is it SyncML?
Paul: Well, SyncML just defines XML DTDs for specific PDA appss. But the first step in terms of real wireless support on PDAs will come in the form of Bluetooth. Motorola is planning to ship their first full Bluetooth suite first quarter next year.
Tony: At Comdex we'll be showing two iPAQs, both with 802.11 wireless LAN cards in them, 100 feet away from each other, sending instant messages back and forth. This is stuff we have here, in house, now. The transport mechanism is Jabber.
Paul: I type in a message in my iPAQ, and it goes from there into the Jabber server and from there to the other iPAQ.
Tony: It's wireless instant messaging in an open-source framework. We're really excited about it.
ELJ: How about Bluetooth?
Tony: That's coming.
Paul: One of the great benefits of Bluetooth is that, in terms of power requirements, it requires a lot less than sticking in an 802.11 card in your iPAQ or laptop. It's very low power. That's one reason it got delayed so long.
ELJ: What about the notion of presence detection, which Jabber includes? Have you guys made something out of that?
Paul: Let's take the example of the weather app again. You could be set up so that every time you go online your presence gets known by the Jabber server. It can say “Hey, I'm here.” And the server can respond accordingly, giving you a fresh weather map, a calendar update or whatever.
ELJ: This kind of brings back the concept of push, doesn't it?
Paul: Yeah, sure. Push technology is used whenever the client is not expecting the data–any sort of alert, such as a news bulletin about a stock that the client is watching. The client has a lightweight message passing service, which will then deliver that message to the application that is interested in it.
ELJ: How does this work, exactly?
Paul: Your client apps register with this ORB–object request broker. And they tell it what XML name spaces they are interested in. Based on whatever namespace that XML packet comes in as, that's what applications it gets sent out to as well.
ELJ: How do you do name-space reconciliation? If you have a lot of different servers and applications running on those servers, each with different memberships in their respective name spaces, with some overlap…I know Jabber conceives your fundamental name as an e-mail address, but that doesn't have to be the case. How do you deal with all that?
Paul: The name space comes with a resource attached to it also. So in our case we might have a kind of a PocketLinux server, so I might be [email protected]/ipaq. I might also be [email protected]/helio, for instance. If I want to send a message to a specific device, I use the same resource mechanism that's already there. Right now with Jabber you can log on, using multiple clients. You can be logged in from work or home. And you can actually have the server direct messages to one of those specifically, while if you take off the resource portion of that address, it will deliver to whichever one is online.
ELJ: So the resource is in charge of what space each name belongs to?
Paul: Yeah.
ELJ: What about privacy and crypto?
Paul: We're kind of relying on the Jabber guys here. In the case of server-to-server connections, they have something in place–I think it's SSL, but I'm not positive. They also have something for using either PGP or GPG, to either a-sign or b-sign and encrypt messages. So it's fairly feasible, especially on a device like the iPAQ, to put a privacy guard on there and use that to talk securely with any of your friends.
ELJ: What about the various cell and phone companies? Have they shown an interest in what you're doing?
Tony: I think I can safely say that many hardware manufacturers from all over the world have expressed more than a passing interest in PocketLinux.
Paul: And in Jabber as well.
Tony: Yes, that's right. If PocketLinux is anything at all, it's the ability to write an infinite number and variation of extraordinarily compelling client apps, extraordinarily rapidly. Of course, interfacing that framework with Jabber creates a system where the whole is greater than the sum of the parts. We're very excited to be working with them.
Paul: The Jabber.com guys are focused on the server. They have instant messaging clients, and instant messaging is a cool application; but Jabber can be used for so much more than that. And that's where our client comes in. PocketLinux is a client for Jabber where you're not limited to just instant messaging. It's a core part of the communications framework within PocketLinux.
ELJ: I had not thought before about the implications of being able to develop a client very rapidly.
Tony: Again, classic instant messaging is a great app, but there is so much more that you can do with Jabber as a generic XML transport. The problem is that we tend to understand instant messaging in terms of a networking paradigm that is perhaps a bit outdated. The networks of the future will be these small devices, always on, always connected and connected at extraordinarily fast rates. The amount of XML data that we'll be able to push down these broadband pipes will be orders of magnitude beyond what instant messaging could deliver even a few years ago.
ELJ: It's so different in kind. That's the thing. It's sort of like extrapolating from telegraph to television.
Tony: That's exactly right.
About the author: Doc Searls is senior editor of Linux Journal and coauthor of The Cluetrain Manifesto. He is also on the Jabber.com advisory board. His opinions are his alone and do not express those of Linux Journal or SSC.
Copyright © 2001 Specialized Systems Consultants, Inc. All rights reserved. Embedded Linux Journal Online is a cooperative project of Embedded Linux Journal and LinuxDevices.com.
This article was originally published on LinuxDevices.com and has been donated to the open source community by QuinStreet Inc. Please visit LinuxToday.com for up-to-date news and articles about Linux and open source.