Interview: Robert Young of Red Hat Software

Getting off the showroom floor turned out to be the real trick, as we were often stopped by people wanting to shake his hand, ask him a question or just tell him how wonderful Red Hat is.

I talked to Bob Young on Saturday, May 22 at Linux Expo. He met me at the Linux Journal booth, and we walked to a quiet place in a nearby plaza. Getting off the showroom floor turned out to be the real trick, as we were often stopped by people wanting to shake his hand, ask him a question or just tell him how wonderful Red Hat is. Even during the interview, we were “found” by one fan who sat quietly and listened until we finished and then jumped up to shake Bob's hand.

Margie: Do you have any numbers on the show?

Bob: No, I don't; you'll have to check with Lisa Sullivan. I actually just arrived at the show; I believe we are doing pretty well.

Our goal, of course, was to keep up our trend, which is to double the show every year. In 1995 at North Carolina State, we had about 200 people; the next year we had about 400 people, again at NC State. This actually wasn't “we” at this point, but a couple of students. Red Hat sponsored it in 1997 at the Biotech Center, and about 900 people came. Last year, the show was held at Duke University and we had about 2000 attendees. So our goal for this year was to attract 4000 people to the show. As of Thursday noon (after the first day of the show), we were at 3000 people. Given that most of the attendees come for the first day of the show, that was probably the bulk. This number does not include vendors or speakers. I think we are well on track to making 4000, but we'll have to wait to actually see what the numbers are.

Margie: It certainly looks like 4000. [Final count was 3600.]

Bob: Oh, good! It's a great show. One of the things I've been noticing is everyone seems to be coming every day.

I've seen a good number of people in the exhibit hall on multiple days—normally, people do the exhibits in the morning, then are done—just coming back to the conferences. But these people are spending a great deal of time with each and every booth.

Margie: That's good to hear. Are you going to keep the goal of doubling each year?

Bob: Absolutely—if not faster! I love to quote Linus' line whenever anyone asks him about the future of Linux: “World domination and fast!” While we may be pleased with the success of 4000 people, it isn't quite world domination.

Margie: 8000 next year—good luck on making it. There have been a couple of incidents that people are talking about. One is the Pacific HiTech t-shirt give-away. Giving out t-shirts is not allowed? I heard they were run off.

Bob: Yeah, this is just show management issues. You'd have to check with Stacy, who is running the show, for more detail. Typically, for vendors to attend a trade show, they buy booths. Vendors who try to sneak into a trade show without buying a booth in order to get the benefit of the trade show—well, whether you go to COMDEX or any of the other shows, you'll be run off pretty quickly if you try to get the benefits of the trade show without contributing to the cost of putting it on.

Margie: That makes sense—how about the poster for LinuxCare?

Bob: That's a funny story, but one we take very seriously. In the end, the only thing Red Hat owns are our trademarks—we don't own any intellectual property—so we have to defend our trademarks very carefully because we can't afford to have them end up in the public domain. Legally, if you don't protect your trademark, you can lose it. We were very surprised by LinuxCare's cavalier attitude. It's one thing to do a parody of someone's advertisement, which the poster was—a parody of the Palm Pilot ad. But it's sort of odd to use a third company's product in that parody. If LinuxCare had put a LinuxCare logo in the woman's hands, that would have been different. But to use our product without our permission as a parody of a Palm Pilot advertisement—well, we were just very surprised they didn't talk to us about it beforehand.

We have a second problem in that Linux users are 99 percent male. The Linux community has not been perceived as a very welcoming community to the other 50 percent of the population. To do a parody that takes an artistic nude from the Palm Pilot ad and turns it into a sexual nude just makes it seem we are going in the wrong direction.

Margie: I truly agree with you on that one. I went by the LinuxCare booth before y'all asked them to remove it. I saw it and told them I found it offensive.

Bob: Well, thank you. It was Lisa Sullivan, our marketing director, who saw it and said, “What are these people thinking?” She went over privately to ask them to stop passing it out because it was sufficiently offensive and tacky, and on top of everything else, it infringed on other people's trademarks. They chose to ask her to talk to their lawyers instead of just complying with the request—so we did. And that's where the story came from.

Margie: That's amazing—I do hate seeing lawyers called. Matthew Cunningham, my cohort here, took a tour of your building and was very impressed. He wanted me to ask you how you decided to use cubicles instead of private offices.

Mr. Young's cubicle, where it all happens

Bob: We actually used private offices in the past. We started with a very open plan; mind you, this was back when we had no money. We couldn't afford cubicles or offices; we just had folding tables. When we moved into our first facility in December 1997, we decided it would be much more efficient if we could close doors and concentrate on the work at hand, so that's the route we went. After working in that environment for a year, we recognized something: while there is a downside to both models, the open space has the one advantage of enabling communication. It's so critically important in our business—in any business that's evolving as quickly as both Red Hat and the whole Open Source movement. We are willing to put up with the distraction of interruptions and noise and everything else you get in an open space in order to focus on the primary benefit, which is the instantaneous communication you can get if you don't have offices.

Margie: Matthew was particularly impressed that you have a cubicle yourself.

Bob: I'm the one who gets cut off the fastest from the information flow around the business. I think it is doubly important that I'm sitting in a cubicle!

Margie: What's going on with 6.0? What's new?

Bob: There are many new things in 6.0. My job isn't so much to make products anymore or come to trade shows and sell them—my job is worrying about the health of the business five years from now. That's why 6.0, as far as I'm concerned, is so important. When you look at the health of Red Hat, you have to worry about the health of the Open Source movement in general.

With 6.0, and I presume it's true for Caldera's and SuSE's and Debian's and Slackware's builds of the Linux operating system, the acceleration in development of this Open Source community had to be considered. What's been happening is we've been getting teams from SAP doing Linux kernel work because they want a faster, better OS for running their application engine. Just recently—this hasn't had an impact in any Linux products yet—SGI released their XFS journaling file system, which is a world-class piece of technology, as Open Source. This is indicative not only of the fact that the open-source development model is working, but also that it is actually accelerating.

In the 2.2 kernel, we are seeing things like the Linux Graphical Network Management tools that compete very effectively with the best management tools you'll find in any of the other UNIX systems, or for that matter, in NT. Of course, we are also getting all the client-side tools, now that we've got both the GNOME and KDE projects rapidly moving forward. The GTK toolkit has become a standard. Netscape reporting all Mozilla code means moving Navigator and Communicator over from Motif to the GTK library. Of course, we already have the GIMP for which GTK was named, and AbiSource is building their word processors against GTK. The GTK libraries, which are under the LGPL, are very exciting because if the client is going to succeed, we have to be able to support graphical applications.

Margie: Each year, Red Hat has gotten easier to install. Now Caldera has their new Lizard install that's really just insert the CD and have it do all the work. Have y'all put in the same sort of easy install that they have?

Bob: We are very impressed with Caldera's Lizard-thing—it's very slick. It has received some very good reviews. We do have certain concerns with our install in terms of going in that direction. Among others, there is always going to be some hardware that is not supported, so you need to be able to back up when you run into a problem. If we build a graphical front end, we want to make sure that during the installation process, you can back up. That aside, you can be assured we are working on a graphical front end to the installation tool set. We also want an open-source one.

We are very interested in what Bodo Bauer is doing with his open-source install shield for his new company, Zenguin. That looks like a very interesting project. I think on the installation side, there is going to be a ton of very rapid advances. This is what is so powerful about this community approach. This is where Caldera is presenting huge value. If it wasn't for Caldera's Lizard install, we might go off and work on something else instead. But with someone pushing us, we're going to have to leapfrog back and come up with a new set of technology. It's that competition in the marketplace that's causing the Linux open-source-based technology to innovate faster than you've seen in the proprietary OS technologies coming out of Microsoft—no one is pushing Microsoft.

Margie: Well, hopefully Linux is pushing them now!

Bob: I suspect you'll see a little more innovation out of Microsoft now.

Margie: Okay, Red Hat has all the big guys giving them venture capital. How did you manage that? Did you go looking for them?

Bob: Yes, we refer to it as “Elephant Hunting”. We went looking for them for a particular reason. In January of 1998, we won InfoWorld's product of the year award for the second year in a row. The first year, we tied with NT; in 1998, they gave it to us outright. We could have been flattered and sat back and said, “Wow, that's pretty cool.” Our reaction was “Houston, we have a problem.” The problem was this: Linux was winning awards everywhere—SuSE in Germany, us in many other marketplaces with many other journalists, and the Apache technology was winning—yet the MIS directors we were talking to were choosing NT anyway, because the MIS directors are looking five years ahead. When they adopt new technology, they have to worry about the investment they have to make in that technology in terms of training and implementation, and they have to worry about getting a five-year life out of it. With the open-source technologies coming from the small companies, they are saying, “Let's see, I can work with a bunch of guys down in the tobacco fields in North Carolina or I can work with the world's most profitable software company, that being Microsoft” and making the easy decision.

The problem is, even if Red Hat doubled in size or even became ten times bigger, we would still be way too small for an MIS director at Ford Motor Company to trust our technology. So we realized the only way around that problem was to get the endorsement of the industry leading players. So that by first getting into Netscape and more recently getting Compaq, Dell and IBM involved, you now have people like Michael Prince at Burlington Coat Factory deploying 1200 Linux boxes he got from Dell, because he could rely on Dell five years from now. So while Red Hat may not be around in five years, clearly Dell Corporation will be.

That was the whole point to it, getting the big ISVs—the Oracles and SAPS on board. Netscape indicated to the ISV community that this was serious technology, and despite the fact that it looks more like the Internet in terms of not being a controlled technology, it nonetheless is a very powerful technology model and has very effective means for ISVs to deploy their technology on these operating systems that are being built on the Linux kernel.

As a result, it's all in the interest of sending the right message both to the ISVs (the industry) as well as to the users—that this stuff is ready to be used. I think we are beginning to see that, in terms of the interest of the industry and the world at large and what we are all trying to do.

Margie: How much are you participating in the Linux Standards Base?

Bob: We are hugely supportive of standards! If you go to the LSB mailing list, you'll find many Red Hat engineers very actively involved. When Dan Quinlan is quoted on this subject, his quotes always seems to have a big question mark in the middle—why are you asking me about this?—because he knows that Erik Troan and Christian Gafton and all our lead engineers are very actively working with him to build the LSB. Yet, we seem to be criticized in the press for not supporting the LSB.

There is only one place where I think we are possibly vulnerable to that criticism and it is unfair criticism. We come out of the Linux world, and one of the big attractions to the Linux world is that you publish all your source code—you publish early and you publish often, as Eric Raymond says. Thus, everything you do is in the public space; you are living in a fishbowl. You don't have the opportunity to do vaporware-type announcements, and we don't. So even when it comes to standards efforts, we don't want the world to think Linux is going to fail, unless a standards body like the LSB succeeds; in other words, what if, for whatever reasons—lack of support by other teams or a standard that is not implementable—it fails? Suddenly everybody writes off Linux, saying “Oh, they couldn't get their act together and do a standard.” We're interested in seeing that standard in place, but we're not willing to bet the future on it. We're going to work very hard, harder than anyone else to make sure of that standard; Red Hat has always adopted things like the Linux file system standards. Everything the Linux community tells us is going to become the standard. glibc was going to become the standard, so we adopted it very aggressively. Then we were getting criticized for adopting glibc, but we were thinking, “Hold on, the whole development community is telling us the future of the technology is glibc.” So you get into a lot of this kind of stuff where we are very focused on staying very close to the development communities. As such, we will not use the media to promote projects or technologies that are not yet real. Just because that's what Microsoft does, and people are so fed up with it that I don't think any of us should go there.

Margie: I agree; I was in programming long enough to know that vaporware isn't a good idea. When Standards do get put in place, how do you think distributions will be able to stay separate?

Bob: It depends on what you're trying to do with Standards. We think Dan Quinlan is going in exactly the right direction. This goes back to the Swiss stock exchange and the MIS director to whom I was talking. He asked me specifically, very pointedly, the question of how do you avoid the standards trap? That is, if you define your standards too narrowly, you limit innovation, because suddenly every player in the industry has to go back to a 30-member standards committee to ask permission to deploy another piece of technology. You must define your standard clearly so that it delivers specific benefits and defines the framework that allows the various Linux engineering teams to build better and better operating systems out of the kernel, and the X Window System and all the pieces that come out of this Open Source community. We define it broadly enough so that the developer is not limited in the innovation he brings to it; then we get the best of both worlds. If we define the Standard badly, so that it's not implementable because it doesn't enable the technology to evolve forward, then all we have successfully done is handed the future of the operating system space back to Microsoft.

So that's the balance and that's why we're saying we're going to support this—we want to see it done properly. But there is the risk, and Dan Quinlan will admit to it, that this thing might not work, and if it doesn't work, we will continue to work very closely through Linux International with “maddog” and everyone else to ensure communication keeps working so our users get the benefit. As I put it, using a car analogy, you need only a single driver's license to drive any kind of car from a BMW to a Hyundai. That is what we want to ensure—that you need only one driver's license to drive any version of Linux.

Margie: You have had a lot of partner announcements, and people have products coming out ported just to Red Hat Linux. Many people are afraid that this is going to cause Red Hat to become the Microsoft of the Linux world. Do you think that is going to happen?

Bob: No. Here's why. We don't do any exclusive contracts. For example, the guys at Metrowerks—on their box, they say it was built for Red Hat Linux. That wasn't our idea—that was their idea for marketing purposes. They recognized, with the partnerships we've been making, that they wanted access to our marketing channels and to send a message to our users that this product was specifically targeted for them. I suspect they will do Code Warrior for Caldera and Pacific HiTech and any other distribution that achieves a decent market size. ISVs are porting to Linux now only because they perceive there are a lot of Linux users. For example, our goal at Red Hat has always been to expand the number of Linux users, not to dominate the Linux space. When we started, Slackware had half a million users—95 percent of Linux users at that time. Today, they have a much smaller market share, probably a million users. Slackware has in fact doubled their number of users in the period we've been around. So we haven't done a particularly good job at squashing our competition! And we have no interest in doing that. Slackware and Caldera and Debian are very much our allies in this effort to make Open Source the definition, to make Open Source a required feature of any operating system the user might consider. If we can do that, we have a real shot at displacing the proprietary binary-only operating systems, whether they are from Microsoft or Apple or other UNIX vendors or IBM.

Margie: I talked to Metrowerks on Thursday, and Mr. Bélanger said they are talking to other people.

Bob: Of course they would, because they would be foolish not to. It's as simple as running make to rebuild it, even if the market they are trying to build it for is, for example, using an older C library. Most of us are using the same C libraries these days, so the stuff probably runs. I don't know this for certain; you'll have to check with the Metrowerks guys, but I suspect it runs on the other distributions just fine.

Margie: They didn't tell me that, just that they would be talking to others. I was talking to Clay Claiborne, a Red Hat reseller, and he told me that Best Buy sells Red Hat for $65 while he has to pay $69.95 for it. How does that kind of pricing work—that companies like Linux Mall and Frank Kasper are charged a higher price than Best Buy?

Bob: It's a real challenge for us. According to U.S. law, you are not allowed to have multiple pricing schemes other than for volume. So we are wrestling with this one as we speak. We have a lot of loyal resellers. The guys at the major shops are quite willing, if they have a product like an operating system—this used to happen with the Macintosh OS at places like MacWarehouse—to sell it at a loss because they know customers will also buy more memory and a hard drive and maybe even another computer. They are quite happy to sell that operating system at a loss in order to get all the additional hardware benefits from it. Also, the guys at Best Buy and Comp USA are competing with each other for retail customers. As far as I can tell, they are not making any money selling Red Hat, but they perceive it is in their best interest to market their products that way.

Margie: Are you saying that Best Buy is paying $69 for it like Frank Kasper and Linux Mall, then selling it at $65 as a loss leader?

Bob: I am the wrong guy to ask, to be perfectly honest. I don't even know what our pricing is! I'd have to send you over to someone on our marketing side to give you precise answers. But yes, that is a problem because they move huge volumes, and as such, they get big discounts and smaller resellers don't get quite the same discounts. All I can say on that point is that we are actively working on it. We have clearly heard the message from the channel and we are going to fix that issue as well as we can.

Margie: Caldera started out as the expensive distribution and has dropped their price, while Red Hat has gone up to $80.

Bob: No, you see, there is complete fallacy in there—we give away our software for free, and we continue to give our software away for free. So our price has gone from zero dollars to zero dollars, and we are hard pressed to see how there is a pricing increase in there. What our price increase relates to is we do have a series of products based around the software you can get for free.

We understand our competition isn't with Caldera or SuSE—our competition is with Microsoft. If we are going to build world-class support organizations and deliver world-class documentation so that customers, who are used to buying products of the caliber Microsoft produces, are going to choose to work with Linux and Red Hat instead, we can't underprice that service. If we do, we won't generate enough money to be able to deliver that quality of service and support to our customers. So our flagship official Red Hat Linux box set is now an $80 product because that's the minimum price we think we can charge to provide world-class support and documentation in that product. That product price may in fact go up in the future if we conclude our customers are actually asking us for it.

Meanwhile, we do have an upgrade version available for $40 on our FTP site called the Core Product which is the full version, and of course even here at the show, you've got Linux Central selling $3.00 Red Hat 6.0 CD-ROMs. I'm not sure we can be accused of being the high-point guys; I think we are the low-cost vendors, but then so is Slackware and so is Debian. As far as we are concerned, we think the guys at Slackware and Debian have much bigger market shares. If we were trying to compete in the Linux space, we would be focused on Slackware and Debian. But we are not trying to compete in the Linux space, we are trying to compete with the very big proprietary OS vendors who have hundreds of millions of customers.

Margie: What can we expect to see from Red Hat in the future?

Bob: Well, hopefully more and more of the same. We think there are huge opportunities to do very big projects in an open-source model. We really like what the guys at AbiSource are doing with their products. We think the guys at Cygnus are going to be doing very clever projects in the open-source space. Of course, we are going to be doing even more projects along these lines.

The enthusiasm you see here at the show has little to do with any product, and everything to do with, for the first time, the user getting real control over the technology they are using. That's the one unique benefit we all deliver to the marketplace. We intend to do more of that and do it better as we generate the resources that enable us to do it. If the customers like it, we'll generate the resources—we're a very customer-focused company. Go talk to some attendees and find out what they want us to do, and that's probably what we are going to do!

Margie: There is a lot of speculation that Red Hat is going to go public. Is that going to happen?

Bob: No comment.

Margie: No comment? Not going to give me a scoop, eh? Well, that was the answer I expected, but I had to ask. [On June 4, Bob announced that Red Hat was going public (] Thanks very much for talking to me.

Touring Red Hat

Red Hat office photos by Matthew Cunningham.


One Click, Universal Protection: Implementing Centralized Security Policies on Linux Systems

As Linux continues to play an ever increasing role in corporate data centers and institutions, ensuring the integrity and protection of these systems must be a priority. With 60% of the world's websites and an increasing share of organization's mission-critical workloads running on Linux, failing to stop malware and other advanced threats on Linux can increasingly impact an organization's reputation and bottom line.

Learn More

Sponsored by Bit9

Linux Backup and Recovery Webinar

Most companies incorporate backup procedures for critical data, which can be restored quickly if a loss occurs. However, fewer companies are prepared for catastrophic system failures, in which they lose all data, the entire operating system, applications, settings, patches and more, reducing their system(s) to “bare metal.” After all, before data can be restored to a system, there must be a system to restore it to.

In this one hour webinar, learn how to enhance your existing backup strategies for better disaster recovery preparedness using Storix System Backup Administrator (SBAdmin), a highly flexible bare-metal recovery solution for UNIX and Linux systems.

Learn More

Sponsored by Storix