Porting LinuxBIOS to the AMD SC520: A Follow-up Report

Getting the board Flashed led to some interesting detective work for the LANL team.
Success

After a little more looking at the manual, the interrupt issues made more sense. As of July 12, we are booting a kernel to multiuser status, and all interrupts are working.

Follow Our Progress

Well, what's left to do? We're going to add VGA support, so we can have a console. We're going to clean things up in the port, too, now that so much is working. There is not space to track the whole process in this article; we do have a word limit, after all. The board is almost done, and once it basically works, we'll go back and really clean the code up, while leaving in bits that we hope will be useful as debug examples for other ports. We intend to build a lot of computers with these boards; they're pretty nice, albeit a little slow. They're great for building portable development and test clusters, however.

We've also discovered, as we frequently do, that the standard BIOS has misconfigured parts in the chipset. This type of misconfiguration is common with proprietary BIOSes, because nobody can check the correctness of everything they are doing. We're going to need to go back and verify all of our settings now, and make sure that no further mistakes were copied from the proprietary BIOS.

Conclusion

LinuxBIOS is a GPLed system you can use to bring up your boards quickly and reliably. It is in use on over a million systems around the world, in applications as diverse as test instruments and televisions. We use it at LANL (Los Alamos National Laboratory) on almost 5,000 cluster nodes, possibly reaching 7,000 by year's end.

In contrast to proprietary BIOSes, LinuxBIOS lets users tailor the system boot-up sequence to fit their exact needs. The Version 2 system we have shown in these two articles is modular and features an object-oriented structure that, in practice, has let us build compact BIOS images. Images smaller than 32KB are routine, even on complex systems such as 8-way Opterons with 32 PCI busses. LinuxBIOS also is portable and has run on 64-bit Alpha systems as well as PowerPC systems. A port to the PowerPC 970, a 64-bit system, is in progress.

For x86 systems, LinuxBIOS includes a full ANSI C compiler, ROMCC, which uses registers instead of memory.

The SC520 is a great chip marred by almost no mistakes. Probably the single biggest problem is the location of the configuration registers, placed right in the middle of the top 2MB of memory, which always should be reserved for BIOS Flash. That said, the more we worked with this part, the more we appreciate its design.

A lot of our testing of register settings for the SC520 was done in Linux before LinuxBIOS. If you start working on your own port, remember the value of doing this type of register probing under Linux--it's a lot easier. To do I/O operations, use the iopl() system call; for memory operations, use mmap().

Principal work on LinuxBIOS in the coming year will be making it easier for engineers to use it who don't have more than a few days a month to work on it. In other words, we're moving from the dedicated, full-time engineer to the engineer who uses LinuxBIOS as a tool but is not that concerned with the internal workings. The overall goal is to reduce the learning curve for LinuxBIOS. Watch the Web page to see the announcements of these changes as they are made.

Acknowledgments

Thanks to Gary Karns and Martin Mayer of Advanced Digital Logic for their help in answering questions, expediting board shipments and providing the information we needed to get this all going.

This research was funded in part by the Mathematical Information and Computer Sciences (MICS) Program of the DOE Office of Science and the Los Alamos Computer Science Institute (ASCI Institutes). Los Alamos National Laboratory is operated by the University of California for the National Nuclear Security Administration of the United States Department of Energy under contract W-7405-ENG-36. Los Alamos, NM 87545 LANL LA-UR-05-5272.

Ronald Minnich is the team leader of the Cluster Research Team at Los Alamos National Laboratory. He has worked in cluster computing for longer than he would like to think about.

______________________

Comments

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

MMCR can be moved...

AndrewD's picture

Quote:
"Probably the single biggest problem is the location of the configuration registers, placed right in the middle of the top 2MB of memory"

You can map the MMCR to any 4k offset in the lower 1G by writing to the CBAR register. 0xFFFEF000 is just the default location, so I would not see that as a flaw.

I ended up using rolo instead of Linuxbios on a SC520 based product I designed a few years ago. U-boot is another very powerful option, but they are both more targeted to embedded products.

MMCR can be moved ... but not removed.

Stefan Reinauer's picture

The problem in this case is that even if you "move" the MMCR, it is still sitting at its old position 0xFFFEF000, in addition to the new one.

Code sample

Craig Ringer's picture

The code snippet is broken - presumably due to mangling by a CMS. Given the extreme difficulty I'm having in getting this godawful commenting system to not mangle these code snippets, that's my working theory. LJ staff, please fix your CMS so it doesn's strip spaces in >code<>/code< blocks! I had to use   in a >code<>/code<block, which is (a) gross and (b) should probably not be interpreted as an entity, but rather as a literal.

In addition to the total loss of indenting, I found two other problems. First:


bios = mmap(0, size, PROT_READ, MAP_SHARED, fd_mem,
off_t) (0xffffffff - size + 1));

looks like it should be:


bios = mmap(0, size, PROT_READ, MAP_SHARED, fd_mem,
           (off_t)(0xffffffff - size + 1) );

and:


#include #
include

should evidently be:


#include
#include

With those changes it builds OK here. The dump doesn't look too interesting on this system - probably not a flash image, anyway - but it's an AMD64 box so it's quite likely the flash is mapped to somewhere different. Any idea what address it might be at?

Webcast
How to Build an Optimal Hadoop Cluster to Store and Maintain Unlimited Amounts of Data Using Microservers

Realizing the promise of Apache® Hadoop® requires the effective deployment of compute, memory, storage and networking to achieve optimal results. With its flexibility and multitude of options, it is easy to over or under provision the server infrastructure, resulting in poor performance and high TCO. Join us for an in depth, technical discussion with industry experts from leading Hadoop and server companies who will provide insights into the key considerations for designing and deploying an optimal Hadoop cluster.

Learn More

Sponsored by AMD

White Paper
Red Hat White Paper: Using an Open Source Framework to Catch the Bad Guy

Built-in forensics, incident response, and security with Red Hat Enterprise Linux 6

Every security policy provides guidance and requirements for ensuring adequate protection of information and data, as well as high-level technical and administrative security requirements for a system in a given environment. Traditionally, providing security for a system focuses on the confidentiality of the information on it. However, protecting the data integrity and system and data availability is just as important. For example, when processing United States intelligence information, there are three attributes that require protection: confidentiality, integrity, and availability.

Learn more about catching the bad guy in this free white paper.

Learn More

Sponsored by DLT Solutions