Ultimate Linux Box: A Work in Progress

Taking advantage of the Opteron blitz to make this year's ULB truly ultimate.

Being as LinuxWorld was last week and some of you have seen the box already, I thought I would let the rest of you in on the surprise we have in store and some changes to the design of the Ultimate Linux Box we had to make based on that surprise.

The surprise is this year's ULB will be a dual Opteron workstation. Early rumors said the Opteron boards would end up being AGP-less for a while, but that turned out not to be the case. The current platform is a pair of Opteron 242's atop an Arima workstation board, which contains an 8x AGP slot, 32-bit PCI slots, optional onboard Serial-ATA interfaces, USB-2 ports and a Broadcom gigabit Ethernet port. I haven't yet had a chance to run the benchmark suite I mentioned last time against the machine, but initial indications say the machine is amazingly fast, due in part to the Opteron's HyperTransport direct, dual-channel memory bus. The one kernel compile I tried before giving up the machine for use at LinuxWorld completed in a stunning four minutes. This particular motherboard has a unique setup; instead of mandating that memory be installed in pairs, as do so many other boards, this one accepts a single DIMM in Slot 1. It simply uses only one of the two HyperTransport channels. Of course, this means your memory access speed is halved, but if you're trying to diagnose hardware issues, it sure does make the job easier.

We had to make two changes in light of the new CPU/motherboard combination. The first was the case; the original Lian Li case I described in "A Case Study" would not accommodate the bigger Opteron motherboard. So Monarch, our supplier, went back to Lian Li and got them to build us a longer, custom case based on the PC-6070 case. It lacks the top fan port of the PC-65B we originally reviewed, but it comes with the same soundproofing as the PC-6070 we reviewed in the noise article in addition to quiet fans. It also proves to be even easier to work on than its little brothers, thanks to the extra room and all those thumbscrews. Changing major components such as power supplies, disk arrays and motherboards is a breeze. The case also comes with a cute little ring clamp device that sticks to the inside floor of the case and keeps all those Serial-ATA cables from getting in the way.

The second change we made was a switch in video cards. On July 22, ATI finally released support for the Fire GL cards for XFree86 4.3, but only for 32-bit Linux and still only in RPM format--but that won't matter, as you will see in a few minutes. NVIDIA, on the other hand, released XFree86-version-agnostic support for the AMD-64 on December 11, 2002. So the ULB this year will come with an NVIDIA Quadro card instead of the ATI Fire GL X1. The Quadro card comes with a new X functionality called TwinView, in which both display ports share a single framebuffer and thus can display 3-D accelerated video on multiple displays, unlike Xinerama. Obviously, this requires quite a bit of mucking about with the XF86Config file, but the README NVIDIA publishes seems reasonably straightforward.

Interestingly enough, all three of the major graphics hardware vendors--NVIDIA, ATI and Matrox--have gone proprietary with their drivers. No longer is there a way to get premium 3-D hardware without resorting to something for which you don't have all the source code. This is a pretty sad state of affairs, which is mitigated only by NVIDIA's obvious dedication to staying ahead of the game--a welcome change from a few years ago. I really wish the graphics people would take a cue from 3Ware and Intel: GPL the drivers and concentrate on making the hardware the best it can be. Unfortunately, I don't see that happening anytime soon.

Obviously a 64-bit computer needs a 64-bit operating system. We have selected SuSE's Linux Enterprise Server 8 for this. Yes, I realize this isn't Free as in Beer, because you can't go out and download it. It'll cost you $767.00 according to the SuSE web site at press time. Remember, the GPL specifically allows you to sell GPL software; we learned that earlier this year in the DansGuardian review. But it is still Linux, still GPL, still Free as in Speech, and for your semolians you get a year of basic (e-mail) maintenance and free updates.

You're still in sticker shock.

Well, I tried what passes for a downloadable x86_64 distribution. It shall remain safely nameless, to avoid embarrassing the vendor. I couldn't get X to run at all. A lot of stuff was broken. In short, it was totally unsuited to be associated with the Ultimate Linux Box. So we're going with SuSE, and pay the price for being on the leading, bleeding edge. I believe the Right Stuff quote is, "No bucks, no Buck Rogers." I know, we would like to believe that Linux is where you can get good, fast and cheap, all on a single CD. Actually, that's still true. It's still cheap, it's just not Free as in Beer. Given that distributions for alternate platforms are often quite different from those for Lintel boxes, I'll be reviewing SLES 8 separately in an upcoming column.

So that's what we've been doing with the Ultimate Linux Box--a new case, a new video card and the Opteron, running the first and what so far is the only publicly available 64-bit-native OS for that processor: Linux.

Glenn Stone is a Red Hat Certified Engineer, sysadmin, technical writer, cover model and general Linux flunkie. He has been hand-building computers for fun and profit since 1999, and he is a happy denizen of the Pacific Northwest.

email: liawol.org!gs

______________________

Comments

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

Re: Nameless distro...

Anonymous's picture

Broken, poorly tested, rushed, low quality....sounds like Mandrake to me...
The difference between a company that tests it's distros in house and fix test and polish before releasing (like SuSE) and companies like Mandrake that basically throw stuff together and dump it on the public for them to test and fix is apparent in this article. Mandrake uses the open source collaboration model to do their work, and then procedes to profit from it. While real companies like RH and SuSE actually pay people to do this testing before hand.
Also, take a look at a SuSE manual sometime, compare it to a Mandrake one. Again, the difference between professionals and amateurs is easily apparent.

It's fine that people love distros like Mandrake, and enjoy using and hacking on them, but it's inappropriate for these same people to push these rough, unpolished distros to new Linux users, and corporate customers. It just causes these people to have a bad taste in their mouth for Linux in general and doesn't further the Open Source cause.
Linux needs to be represented to these kinds of uptight users by grownups, with grownup products. Not by elite wannabe types screaming about how their Mandrake/Enlightenment desktop with the haxorz theme rules!

ULB with SLES8 is a fine representation of how things can be for people having the guts to stray off the Windows/Intel path.

Re: Ultimate Linux Box: A Work in Progress

Anonymous's picture

"The HyperTransport (HT) buses are basically like very fast PCIX..."

HT came first. PCI-X is HT-like.

Re: Ultimate Linux Box: A Work in Progress

Anonymous's picture

If people want to test 64 bit Linux simply run it on the P Mac G5. It can run 32 bit software too. And try Red Hat's libraries for the AltiVec math CPU in the G5. Interresting to see a test betwwen the PPC 970 and AMD 64!

--
Mattias Andersson, Falun Sweden
On a Mac 8500, Yellow Dog Linux

Re: Ultimate Linux Box: A Work in Progress

Anonymous's picture

There is a quite obvious link between the SuSE Linux people
and the ATI Linux developers. SuSE had ATI (better say
FireGL) drivers included on their Linux distribution for several
years now. Any such outraging project for getting the best,
fastest and latest hardwar run on Linux should considered
talking back to the expert people of really getting the best
that they can offer you - you just have to ask them,
but i strongly suppose you have not even tried it.

Re: Ultimate Linux Box: A Work in Progress

Anonymous's picture

Can you play 'pong' with this setup?

Re: Ultimate Linux Box: A Work in Progress

Anonymous's picture

Forget Pong. How well does it run vi?

Re: Taroon

Anonymous's picture

Did u try Taroon?

Re: Ultimate Linux Box: A Work in Progress

Anonymous's picture

The nameless x86-64 distribution is likely gingin64, Red Hat's technology preview, based on Red Hat Linux 9. It might have been SuSE Linux 8.2 beta or Red Hat Enterprise Linux 3.0 beta, but they haven't been out as long.

We've got one Opteron system at work, but it's tied up with 32-bit Windows and Linux compatibility testing. Eventually, we'll use it to test SuSE Linux Enterprise Server 8 and Red Hat Enterprise Linux 3.0. I can't wait for the Athlon 64 launch this Fall.

Re: Ultimate Linux Box: A Work in Progress

Anonymous's picture

Droool.....

Re: Ultimate Linux Box: A Work in Progress

Anonymous's picture

"and for your semolians you get ..."

I think that should be simoleons.

Ow! stop smacking me! I'll quit nit-picking!

Re: Ultimate Linux Box: A Work in Progress

Anonymous's picture

I'm a systems architect and have been studying the Opteron for a while now. You have a misconception that I'd like to clear up.
The HyperTransport (HT) buses are basically like very fast PCIX buses and have nothing to do with the processor's access to memory. In fact, some Opteron CPUs have three HT buses, but only support dual interleaved memory configurations (requiring two DIMMs). You talk about how a system with one DIMM "simply uses only one of the two HyperTransport channels" but this is wrong. Both HT buses work fine, but only one of the memory channels is in use. In fact, all Opteron CPUs have this ability to use one or two memory channels. The BIOS is what makes configurations of the memory channels flexible enough to handle one DIMM or two. Apparently some BIOSes do allow this and some don't.

One more tidbit. In addition to the PCIX like functionality provided by the up to three HyperTransport buses, these buses can also serve as cache-coherency buses for the CPUs in the system. This means that all CPUs can share all of each others' memory in a way that allows their internal caches to transparently speed up accesses without ever seeing stale data due to the use of these caches. The CPU to CPU bandwidth over one HT bus peaks at 12.8Gb/s in each direction (assumes 800MT/s HT). This results in a hella-fast CPU to CPU communications path that will eventually cause the rethinking of a lot of strong assumptions in systems architecture about how symmetric multiprocessing systems should be built. It's very flexible and very very exciting.

White Paper
Linux Management with Red Hat Satellite: Measuring Business Impact and ROI

Linux has become a key foundation for supporting today's rapidly growing IT environments. Linux is being used to deploy business applications and databases, trading on its reputation as a low-cost operating environment. For many IT organizations, Linux is a mainstay for deploying Web servers and has evolved from handling basic file, print, and utility workloads to running mission-critical applications and databases, physically, virtually, and in the cloud. As Linux grows in importance in terms of value to the business, managing Linux environments to high standards of service quality — availability, security, and performance — becomes an essential requirement for business success.

Learn More

Sponsored by Red Hat

White Paper
Private PaaS for the Agile Enterprise

If you already use virtualized infrastructure, you are well on your way to leveraging the power of the cloud. Virtualization offers the promise of limitless resources, but how do you manage that scalability when your DevOps team doesn’t scale? In today’s hypercompetitive markets, fast results can make a difference between leading the pack vs. obsolescence. Organizations need more benefits from cloud computing than just raw resources. They need agility, flexibility, convenience, ROI, and control.

Stackato private Platform-as-a-Service technology from ActiveState extends your private cloud infrastructure by creating a private PaaS to provide on-demand availability, flexibility, control, and ultimately, faster time-to-market for your enterprise.

Learn More

Sponsored by ActiveState