Bringing Linux Appliances to Market
Early in January, Intel announced plans for new Intel-branded Internet appliances that would run on Linux. (The press release is here.) They also showed and demonstrated some of these at their booth at CES--the Consumer Electronics Show--in Las Vegas. It was there that I got to play around briefly with two devices: a computer-like thin client, sitting in an ersatz kitchen, and a TV set-top box in an ersatz living room. Both were beyond impressive. They were scary.
The web box had a display that seemed to have more in common with an airport kiosk than a computer of any kind. Fundamentally, it was a browser. A Mozilla browser, in fact, but without the usual windowing features. It was simply a window on the Web, with a set of other handy features: e-mail, sticky notes, calendar and so forth. It was equally open and proprietary: on the one hand, it used open-source software (Linux and Mozilla); on the other, it was a closed and therefore proprietary box. It was, literally, an appliance.
The set-top box looked like, say, a DISH or DirectTV box, but with one important difference: when you thought you were looking at just another proprietary cable company display, you were looking at a browser rendering HTML. Intel had webified The Tube by putting a browser (again, Mozilla) interface on all the potential choices a user might face. It had even moved some of the directional buttons (e.g., forward and back) from the browser window to the remote control. Again, it was that strange mix of open and closed.
I wondered: what would we call these boxes? LinTel? MozTel? MoLuxTel? Innux? Didn't matter: Obviously, "Intel" was the only name that mattered. Linux and open source were nothing more than a better building material and a better building method. In fact, Linux and Mozilla disappeared into the plastics, the flash memory, the deep background. This is what we mean by "embedded", folks.
Intel has already been aggressive about moving these things. In the U.S., they have signed up US West and Lucent. Overseas, they have signed up NEC's BiGlobe in Japan, France's LASER, which is part of the Galeries Lafayette Group, and Nokia. Reported costs to service providers range from about $300 to $700. But cost of the devices to end users will depend on how service providers price and promote the things. In some cases, the price will be zero.
At CES, I had a long talk with Jon Bork, General Manager of Intel's Broadband Services Division. Jon told me he led a team that was instrumental in selecting Linux and driving its adoption within the organization. Some of what he said was so interesting that I had to sit down and talk with him again, this time with a tape recorder running. Here is the conversation that followed.
Doc Searls: Let's start with why you chose Linux and what you're trying to do with your appliance strategy.
John Bork: The decision to use Linux, in both the web appliance and the set top, was driven by a multifaceted evaluation. We looked at business issues, technical issues, and where Intel had the ability to innovate quickly, among other things.
Doc Searls: What other OSes did you evaluate?
John Bork: Windows, Windows CE, X Works, and Linux, among others. But we chose Linux because it offered the best balance across a number of factors. One was that we thought it had the right footprint for the problem--a small one. Another was cost. Certainly the price was right. We needed to be able to make changes as the market changed along with us, and do it quickly.
Doc Searls: Did Linux come up short in any way?
John Bork: Yes. All of them did. With Linux, it was in areas like USB support. With X Works, there was a problem in getting the full family of browser plug-ins. With Windows CE, we didn't see a fast enough rate of innovation. Also, we need to get in and make bug fixes quickly, and we were limited with CE. And one of the big issues with Windows is that it's not an appliance-oriented OS. But at the end of the day, we looked at all these factors and talked to our customers, and our customers said Linux is the right tool for solving this problem. The most interesting fact was that our customers were asking for Linux, having studied the tradeoffs themselves.
Doc Searls: Tell me more about the products you are putting Linux in, and how you expect them to evolve in the marketplace.
John Bork: We're addressing a couple of different market segments. We have what we call the Web appliance internally. It's a single integrated appliance, meaning a display, the communication piece and the processor are all in one box. The Web appliance has integrated telephony, meaning either a wireless or a wired handset. And the intent is to bring Internet capabilities into parts of the home that you would traditionally not see the Internet. And more than that, because of the telephony, it's to integrate the telephony services in a way that makes the telephony easier to use. For example, it's difficult to remember what "#67" does if you don't use it all the time. But if you put it on the screen so the guy just hits a button for Caller ID or Last Caller Redial or whatever, now you have really made it easier for those services to be used. And of course, to the people we're selling to, such as the telcos, that's very attractive because that's incremental business for them.
Doc Searls: There's directory integration.
John Bork: That's another one. If you think about all the opportunities of telephony and Internet capabilities coming together, there are all kinds of opportunities here.
Doc Searls: In a way, it turns your Net into your PBX. All displayed by HTML.
John Bork: That's one way to see it. I think the biggest opportunity the Internet brings is crossing it with other technologies and market segments where it's never been thought about before. We're doing that with telephony and television--
Doc Searls: --at the client.
John Bork: Yes. Other people back in the infrastructure are doing things like integrating satellite and the Internet.
Doc Searls: A lot of people are working on voice over IP.
John Bork: Right. But just think about your traditional telephone handset in the home, and all the services hidden in that 12-button pad which you can't use because it's too big a bother. Even there, we see a huge opportunity. Look at things like Caller ID, which often requires an outside box with a one-line display. That service improves tremendously when it's integrated into a web appliance.
Doc Searls: Suddenly, you have a phone top.
John Bork: Yeah. Absolutely.
Doc Searls: Are the products you showed at CES just prototypes?
John Bork: Well, we're going beyond that. The intent in the web appliance space is to bring an Intel-branded product to market, working with the telcos--both the CLEC and ILEC type customers* --to provide not just the hardware but the client software and the server software, so it's a complete end-to-end solution.
Doc Searls: So your customer resells branded Intel boxes. Not just "Intel inside", but Intel from the silicon to the plastic.
John Bork: Absolutely. And the intention is also that Intel's work would be apparent in the software; so that the telcos, the ILECs and CLECs, would have their primary brand at the portal.
Doc Searls: How about somebody like Panasonic, which is a big supplier of PBXes to the SOHO (Small Office & Home Office) market (and one of whose impossible-to-program PBXes we are talking over right now)? What happens when you throw the Internet into that market? Would a Panasonic want to private-label something like this?
John Bork: Every discussion is unique. I think if you step back from the Home Products Group (HPG) and look at Intel's strategy, you see we intend to be the building block supplier to the Internet economy. Insofar as Panasonic sees a play using Intel building blocks, we would of course be very interested in having that conversation.
Doc Searls: Telephony is one, and television is another. But do you guys see your stuff in other existing appliances? At the Transmeta rollout, people were talking about the browser on the door to the refrigerator--with a bar code reader in the back, an inventory database and a keypad for ordering more groceries or doing whatever else might make sense. Do you see yourselves in that kind of market as well?
John Bork: Not exactly...
Doc Searls: ...Okay, then what happens when the browser is the interface for nearly everything?
John Bork: That's the hundred-billion-dollar question. As everybody works through this new appliance model, it will start to get clear. In the wireless space, you see palms, pagers and cell phones all starting to integrate. At Intel, we have a whole group just focused on wireless. You see new peripherals to the PC, where the PC is the processing hub and there are wireless connections to tablets and appliances throughout the home: from hand pads to fridge pads. I think the real business question is cost structure. The big cost drivers are the batteries and the LCDs. That has nothing to do with the microprocessor or the browser. How many different devices will a customer buy with an expensive LCD and batteries that need constantly to be replaced or recharged? The model for the Internet is now to pay for services. What services will I pay for that involve interacting with my refrigerator? And how much will I pay for that in addition to what I pay for my telephone, PC, Web appliance, palm appliance, cell phone? Two years ago, I heard that if you added up all the services people would like you to consume, the total would be more than two thousand dollars a month. And the truth is that consumers are willing to spend about forty to sixty dollars a month.
Doc Searls: What other market segments are you looking at?
John Bork: There are two. One is building a cable modem-attached set-top box. The concept here is bringing the Internet to the television set. In some areas of the world, the cable-attached approach of bringing Internet service in over high-speed hybrid fiber coax (HFC) is the right solution. That's where you run the neighborhood in fiber and then copper coax into the home. In many parts of the world, because they started later than in the U.S., they are farther along in their fiber deployment. Then the third market involves a partnership with Nokia, focusing on satellite boxes. That's Internet delivered by satellite, with a POTS (plain old telephone service) back channel. We think there is a tremendous opportunity for the direct-to-home market.
Doc Searls: So who are your customers, so far?
John Bork: On the web appliance side, we have a couple. One is LASER, which is Lafayette Services, in France. They're part of Galerie Lafayette, one of the biggest retailers in Europe. If you do a lot of business as a customer with Galerie Lafayette, eLaser may give you this appliance if you spend more than a certain amount of money. That's one model. Another is what we have with BiGlobe in Japan, a subsidiary of NEC. They private-label the appliance and bundle services. On the set-top side, we have an agreement with Hughes Network Systems to provide support for AOL/TV or DirecTV. Then we have an agreement with Nokia for a set-top platform based on Linux. We haven't announced a service provider for that one yet. Finally, in China, we have a deal with Pacific Century Cyberworks, PCC, to bring the Internet to people over there over a modem-attached set-top box. And we are working on other things that are yet to be announced in other parts of the world.
Doc Searls: With that Galerie Lafayette deal, it seems to me what you've got here is the Minitel replacement, at least for France. People outside France might not be familiar with the Minitel, but it's a terminal attached to the telephone system, and it was very successful for a very long time. So I imagine that this appliance would be easy for French people to adopt as a concept.
John Bork: That's true. When we were searching for analogies, the first thing out of many people's mouths was, "It's a Minitel." We said yes, exactly, that's a perfect model for what we're trying to do here. But this appliance is based on open standards and open source, rather than proprietary in the way the original Minitel was.
Doc Searls: So you've got home appliances, set-top boxes and satellite boxes.
John Bork: Think of them as very similar product concepts with very different communication paradigms.
Doc Searls: At the software level, you've gone with two open-source solutions: Linux for the OS and Mozilla for the browser. And you like them, because they are handy and compact building materials that you can apply any way you like.
John Bork: Right. We're also very active with the Mozilla development team, because one of the things we found as we got engaged was that we needed to go fast, and we could get that way by becoming part of the Mozilla development team.
Doc Searls: What have you contributed there?
John Bork: Stuff like foreign language support. We needed Chinese, for example. Internationalization is customarily dealt with later in the development cycle, after you get the core product out. We needed it earlier.
Doc Searls: How about items like USB support? I notice USB interfaces on the appliances you had on display at CES. USB and Infrared.
John Bork: Yes. We have been very active in USB driver support. If you look at where Intel wants to take the PC, you'll see we're moving toward legacy-free goods. On the PC, that means USB, 1394 (FireWire) and PCI-type connections into the box, to remove some of the ISA, serial and parallel port kinds of stuff, which was where a lot of the reliability issues showed up. On the appliance, we think USB is one of the most powerful attach mechanisms for cameras, printers, microphones, scanners and other devices you might want to have around, say, a set-top box, to enable an Internet experience. So we have been very active in USB driver development in the Mozilla movement. We're actually among the people reviewing the changes coming in to the driver, and helping to drive those changes. This is also complementary to what we're already doing on many other OS platforms.
Doc Searls: You invented USB.
John Bork: Yeah.
Doc Searls: What I like about what I'm seeing with USB is that it describes the size and shape of the TinkerToy holes you can use to build all kinds of neat stuff.
John Bork: It's a very powerful way to bring peripherals into an overall configuration.
Doc Searls: How about FireWire?
John Bork: It's one of those technologies where we're waiting to see what the market does. As you know, Intel is already working on USB2, which is a higher-speed USB connection. But the customers have the final say here. Early prototypes had 1394 and customers said "We don't need it. Take the cost out."
Doc Searls: I was impressed with the set-top box implementation of Mozilla. What the viewer saw looked like any selection screen with DISH or DirecTV, where you are looking at a variety of program and other choices in a table or some other visual format. But what you are looking at is HTML. What you've done is moved a lot of the desktop browser interface onto the remote control. Forward, back, stop.
John Bork: If you think of TV as just another browser window, you're exactly on the concept.
Doc Searls: Except that a TV isn't a computer, nor is a set-top box. What did you do, technically, to implement this in an appropriate way?
John Bork: We had some very interesting challenges in bringing out a set-top box that really is able to access the Internet. Three in particular. The first was ATVTF, a standard for bringing the TV into the browser, and also for enhancing the TV image with other pieces, such as hot spots or live-click technologies, where basically TV and the Web become a seamless experience. The second was enhancing the set-top box for TV. So we spent a lot of time working on TV quality out. How do we produce a product that really makes the Net look best on the TV set? Traditionally, PC architectures have not been optimized for that design goal. They have been optimized around a progressive scan monitor on your desk. The third piece was designing a GUI and an interface that looks good in a living room environment: a 10-foot rather than a 2-foot experience.
Doc Searls: It seems to me that the two experiences are coming together. I was impressed at CES by how much the display technologies seem to be merging. You certainly see that already when you slap a DVD into your laptop and watch a movie.
John Bork: There is a technology rathole here--
Doc Searls: --We like those. We're a technical publication.
John Bork: Good. It's interesting. Traditionally, Intel loves high MIPS-consuming applications. So things like DVD movies, we love that stuff. The interesting part is that the computer is better able to mimic the TV than the TV is able to mimic the computer. Because with a computer, you've got the processing power and a progressive-scan high-quality display. The biggest challenge is bringing TV up to the display resolution and processing power that the computer has today. So when you look at some of those huge plasma TV screens we saw at CES, and go "Wow!" when they show a DVD, we forget we're still dealing with less resolution than you get on a computer. Another issue: web content with a white background looks pretty crummy on a TV. A computer display has no problem displaying pure white. But if you talk to a TV tube manufacturer, they tell you they optimize for a gray balance in a picture tube. A solid white background would burn out the tube. There are also some interesting technical issues regarding our current NTSC-interlaced scan television sets that are exciting to me. You're going to start seeing progressive-scan TVs, high-resolution TVs and other technologies that the TV manufacturers will bring out. Waiting for those is actually the biggest barrier to making the computer and the TV an equivalent experience in equivalent usage paradigms.
Doc Searls: What we saw with WebTV was making a bird look like a dinosaur. They did some neat type and layout tricks, but ironically, the system choked at places like Microsoft's own home page, which has acres of detail with lots of tiny type.
John Bork: The type is an issue. We think WebTV has struggled with conveying a full Internet experience. You need the ability to have all the relevant plug-ins there, so that when the user goes to a web page, they don't get "Go to your PC to see what this really looks like."
Doc Searls: It was also interesting to me that with your implementation of Mozilla, some of the browser buttons were now at the physical layer: on the remote control. They actually seem to belong there.
John Bork: We went through a big exercise in '99 where we went out to consumer electronics stores, counted buttons on remotes, compared usage models, and asked, "What are people really going to use?" You never want to present a user interface through a 100-button remote. The reality is that people don't use them. So the challenge is to provide a web experience through a remote with 30 buttons or less. We gave ourselves a 'button budget' and were very stingy about using more of them. The question constantly was, "How do you get all the functionality out of a very simple and easy-to-use remote?" If you have complex, hidden or hard-to-use functions, don't put them on the remote. Allow the user to pick them off the screen.
Doc Searls: Let's go deeper into the technology: how you hacked on Linux and Mozilla, and how you ignored legacy, as you put it, to develop these new products.
John Bork: We started with the Celeron processor, basically because of the cost points we wanted to hit in the appliance market segment. Outside of that, we wanted to optimize for delivering a sealed box with a low-rise form factor. That meant using a PCI bus, because the communications piece is still somewhat of a moving target. We told ourselves we'd love to jump to DSL, but the market isn't there yet. So we're working with both POTS and DSL on the web appliance. On the set-top box, we needed to design for both POTS and satellite, alongside cable modem technology, all in the same box, so we don't have to build the whole thing from scratch each time. TV tuner technology is right there on the board, leaving legacy I/O out and putting USB in. So what you start with is a PC and end up with a device that harnesses PC power, but optimized to solve different problems. This meant peeling away irrelevant pieces of the PC architecture, and then adding other pieces, like TV tuning and telephony, that are core to the product concept.
Doc Searls: From a Linux perspective, you peel away the legacy operating system and substitute one that is much smaller and more adaptable.
John Bork: Yes. Windows CE is an environment where Microsoft says, "this is what the reference design looks like, and as long as you build this, Windows CE will work on it." Linux, on the other hand, is an erector set, where we design the reference hardware to meet the problem we're trying to solve and then go to Linux for the piece parts from the bin to build exactly what works.
Doc Searls: And there are fewer licensing issues.
John Bork: There are three things involved here. First, can you get the license? That's not our biggest concern. Second, can you innovate fast enough to get the right product out of it? And third, given that we're all new here, and the successful formula is anybody's guess, can we change and fix things at an accellerating pace? We chose Linux based on the ability to innovate and fix things quickly, more than "Can we go negotiate a license?" Intel's position is OS-agnostic, really. Different problems require different OS solutions. There isn't one OS for every problem. But in this case, our analysis said that Linux is the right OS to solve this set of problems.
Doc Searls: And I imagine there are more and more people inside Intel who know how to build solutions with Linux.
John Bork: Yep. Absolutely. We not only have Windows experience, but also a long history of UNIX experience. So when we went to work on Linux, there were lots of guys who said, "Hey, I've been working on similar problem sets for a very long time. We can help." That makes hiring both inside and outside of Intel a very easy job.
Doc Searls: How do you cope with open-source development inside a company like Intel, which, like most companies, has an employment agreement that says "everything I do when I work at the company belongs to the company"?
John Bork: We set out to solve three problems here. Number one was, for the things we identify as our intellectual property, how do we make sure they don't seep out? Second, for the things covered under an open-source license, like a GPL, how do we make sure they do seep out? There, we need to make sure they seep out in a way that works with the spirit and the letter of the license. Third, how do we enable Intel employees in their own time to work on open-source projects? In the past, we've always dealt with operating systems that were owned by somebody. Now we're in this new territory. We're in the Linux community, and we need to do the right thing both for ourselves and for that community. So we've taken a couple of steps. First, HPG (the Home Products Group) has an open-source manager whose full-time job is to design, drive, audit and manage these processes.
Doc Searls: Is this spreading around the company, or is it confined to the HPG?
John Bork: Right now, the open-source manager works for the Home Products Group, but what's interesting is, as he has gone out and trained other parts of Intel on open-source processes and on our policies and procedures, other groups are now starting to hire his equivalents. I would say we started it, but it's spreading pretty rapidly as Linux begins to solve other problems.
Doc Searls: Where is the HPG?
John Bork: At three sites. We have a division in Oregon. Our general manager and VP/general manager work in Santa Clara, where we have a small group. Then the remaining half or two-thirds are in Chandler, Arizona.
Doc Searls: Okay, let's get back to how open source is working at Intel.
John Bork: Yes. So, out of this process has come new corporate policy that allows people to actively participate in the Open Source community. That's the first thing. Second, we've put policies and procedures in place that govern the way we write and tag the software, and build the software, identifying what's ours and what's the community's. In the latter category, there is lots of software that we want to make sure gets back into the community. So we've put training in place, where all of our engineers learn the differences and how to work with those differences. There are different models involved. With Mozilla, for example, our work is immediately going back into that Open Source community. In other places, we are innovating mostly internally, and the time to bring the work out into the community is when we ship product. There, we will have validated, tested, and found it's ready to be submitted. But it's been a big challenge, because this is a new paradigm for a large corporation like ours. Open source is a new phenomenon for us.
Doc Searls: In many ways, however, open source is really the natural way things work. In building construction, for example, if somebody thinks of a better way to raise a roof or a floor, they're going to share that with the next crew they work with. So word spreads. The knowledge of how to build something tends to be shared. And it's in the interest of everybody that knowledge sharing takes place. But it's new to the software business, where pre-fab construction out of large pieces in which the designs belong to one big company has been the norm for a long time. But in the Open Source world, anybody can innovate and share it with everybody. This is good, but it can also be scary for a company that doesn't want the design changed out from under them on a big project.
John Bork: The constant challenge for big companies is in creating and adopting innovations as fast as possible. This has never been easy, but I do believe innovation is the answer. While big-company lawyers may try to slow that down a little bit, the working purpose today is to go as fast as you can. The PC Internet economy has shown us conclusively that the guy who wins isn't the guy who holds up the train. It's the guy who makes it go faster than anybody else.
Doc Searls: That's hard for a big company.
John Bork: As a big company, your first instinct is to name three guys, segregate them into a corner, firewall them in and tell them to work on a solution. It took us a few minutes of staring at this problem before we realized that was not the way to go here.
Doc Searls: That says good things about your culture. There are still lots of companies that would act on that first instinct. They would isolate a team around a problem, rather than recognize that the team needs to be part of a larger community of shared interests, and that sharing will help solve the problem faster. In fact, at the Transmeta announcement, I was the one who asked the obvious question: "Why the secrecy?" They may have achieved a publicity goal, but at what cost? How could it have helped to deny their engineers the opportunity to converse outside the company about the innovations they were working on, especially if it invited design-ins with long development cycles?
John Bork: Intel's got a neat culture that's really confrontive. We had to sit down and say, "How were we going to make this doggone thing work?" and, "What are we going to have to do differently?" The cool thing was our culture is flexible enough that when we have to change, we do. The opportunities are what matter most. With guys like Andy Grove and Craig Barrett driving, we change pretty fast.
Doc Searls: How much of the kernel are you using?
John Bork: We just take the source off the Net, pull out the stuff we don't need, seeing what libraries are used and not used, and constraining ourselves from there. See, the biggest challenge we face is getting the OS, the browser, the plug-ins and a few applets into eight megabytes of Flash. So a lot of our work has gone into analysing what pieces we need, and how we skinny this thing down.
Doc Searls: What about streaming plug-ins?
John Bork: We are in the process of negotiating with the major streaming plug-in manufacturers. Again, depending on the marketing segment, there are clearly different needs for streaming media. In the set-top box segment, of course, streaming content is the holy grail. It turns out that in a Flash-based box, you can get most of what you want with a browser and plug-ins. You're not going to be able to store that content. You'll show it and move on.
Doc Searls: How do you deal with plug-ins that change over time?
John Bork: One of the things we focused on is that the box needs to be software-downloadable, down the wire from the service provider, and manageable from the service provider. And that's critical because the Internet will continue to evolve at lightning speed. And it is absolutely essential that as new plug-ins come along, we can download them in the field without rolling a truck, which is the traditional way in which service providers have handled upgrades. The proposition is to keep up with the Internet's own rate of innovation.
Doc Searls is Senior Editor of Linux Journal
Practical books for the most technical people on the planet. Newly available books include:
- Agile Product Development by Ted Schmidt
- Improve Business Processes with an Enterprise Job Scheduler by Mike Diehl
- Finding Your Way: Mapping Your Network to Improve Manageability by Bill Childers
- DIY Commerce Site by Reven Lerner
Plus many more.
- diff -u: What's New in Kernel Development
- Server Hardening
- Giving Silos Their Due
- 22 Years of Linux Journal on One DVD - Now Available
- What's New in 3D Printing, Part III: the Software
- Controversy at the Linux Foundation
- Don't Burn Your Android Yet
- February 2016 Issue of Linux Journal
- Firefox OS