Porting LinuxBIOS to the AMD SC520: A Follow-up Report

Getting the board Flashed led to some interesting detective work for the LANL team.

After a little more looking at the manual, the interrupt issues made more sense. As of July 12, we are booting a kernel to multiuser status, and all interrupts are working.

Follow Our Progress

Well, what's left to do? We're going to add VGA support, so we can have a console. We're going to clean things up in the port, too, now that so much is working. There is not space to track the whole process in this article; we do have a word limit, after all. The board is almost done, and once it basically works, we'll go back and really clean the code up, while leaving in bits that we hope will be useful as debug examples for other ports. We intend to build a lot of computers with these boards; they're pretty nice, albeit a little slow. They're great for building portable development and test clusters, however.

We've also discovered, as we frequently do, that the standard BIOS has misconfigured parts in the chipset. This type of misconfiguration is common with proprietary BIOSes, because nobody can check the correctness of everything they are doing. We're going to need to go back and verify all of our settings now, and make sure that no further mistakes were copied from the proprietary BIOS.


LinuxBIOS is a GPLed system you can use to bring up your boards quickly and reliably. It is in use on over a million systems around the world, in applications as diverse as test instruments and televisions. We use it at LANL (Los Alamos National Laboratory) on almost 5,000 cluster nodes, possibly reaching 7,000 by year's end.

In contrast to proprietary BIOSes, LinuxBIOS lets users tailor the system boot-up sequence to fit their exact needs. The Version 2 system we have shown in these two articles is modular and features an object-oriented structure that, in practice, has let us build compact BIOS images. Images smaller than 32KB are routine, even on complex systems such as 8-way Opterons with 32 PCI busses. LinuxBIOS also is portable and has run on 64-bit Alpha systems as well as PowerPC systems. A port to the PowerPC 970, a 64-bit system, is in progress.

For x86 systems, LinuxBIOS includes a full ANSI C compiler, ROMCC, which uses registers instead of memory.

The SC520 is a great chip marred by almost no mistakes. Probably the single biggest problem is the location of the configuration registers, placed right in the middle of the top 2MB of memory, which always should be reserved for BIOS Flash. That said, the more we worked with this part, the more we appreciate its design.

A lot of our testing of register settings for the SC520 was done in Linux before LinuxBIOS. If you start working on your own port, remember the value of doing this type of register probing under Linux--it's a lot easier. To do I/O operations, use the iopl() system call; for memory operations, use mmap().

Principal work on LinuxBIOS in the coming year will be making it easier for engineers to use it who don't have more than a few days a month to work on it. In other words, we're moving from the dedicated, full-time engineer to the engineer who uses LinuxBIOS as a tool but is not that concerned with the internal workings. The overall goal is to reduce the learning curve for LinuxBIOS. Watch the Web page to see the announcements of these changes as they are made.


Thanks to Gary Karns and Martin Mayer of Advanced Digital Logic for their help in answering questions, expediting board shipments and providing the information we needed to get this all going.

This research was funded in part by the Mathematical Information and Computer Sciences (MICS) Program of the DOE Office of Science and the Los Alamos Computer Science Institute (ASCI Institutes). Los Alamos National Laboratory is operated by the University of California for the National Nuclear Security Administration of the United States Department of Energy under contract W-7405-ENG-36. Los Alamos, NM 87545 LANL LA-UR-05-5272.

Ronald Minnich is the team leader of the Cluster Research Team at Los Alamos National Laboratory. He has worked in cluster computing for longer than he would like to think about.



Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

MMCR can be moved...

AndrewD's picture

"Probably the single biggest problem is the location of the configuration registers, placed right in the middle of the top 2MB of memory"

You can map the MMCR to any 4k offset in the lower 1G by writing to the CBAR register. 0xFFFEF000 is just the default location, so I would not see that as a flaw.

I ended up using rolo instead of Linuxbios on a SC520 based product I designed a few years ago. U-boot is another very powerful option, but they are both more targeted to embedded products.

MMCR can be moved ... but not removed.

Stefan Reinauer's picture

The problem in this case is that even if you "move" the MMCR, it is still sitting at its old position 0xFFFEF000, in addition to the new one.

Code sample

Craig Ringer's picture

The code snippet is broken - presumably due to mangling by a CMS. Given the extreme difficulty I'm having in getting this godawful commenting system to not mangle these code snippets, that's my working theory. LJ staff, please fix your CMS so it doesn's strip spaces in >code<>/code< blocks! I had to use   in a >code<>/code<block, which is (a) gross and (b) should probably not be interpreted as an entity, but rather as a literal.

In addition to the total loss of indenting, I found two other problems. First:

bios = mmap(0, size, PROT_READ, MAP_SHARED, fd_mem,
off_t) (0xffffffff - size + 1));

looks like it should be:

bios = mmap(0, size, PROT_READ, MAP_SHARED, fd_mem,
           (off_t)(0xffffffff - size + 1) );


#include #

should evidently be:


With those changes it builds OK here. The dump doesn't look too interesting on this system - probably not a flash image, anyway - but it's an AMD64 box so it's quite likely the flash is mapped to somewhere different. Any idea what address it might be at?