Linux on Alpha AXP—Milo, The Mini-loader
Late in 1994, I was in the US visiting my home group in Hudson, Massachusetts, where the Alpha AXP processors are built. On a free morning, I followed up a rumour that I had heard a few weeks before. The rumour was that Jim Paradis was porting Linux to Alpha. At that time I did not know anything about Linux other than that it was a freeware version of Unix developed by a student in Finland. When I caught up with Jim, I was in for a surprise. Well, two surprises actually: Linux—running on an Alpha laptop. We chatted for a while and I soon became infected (if that's the right phrase) with Jim's enthusiasm.
One subject that we discussed was Linux/Alpha's need for a small loader. On an Intel PC system, the firmware that initialises the system when it is powered on is known as BIOS. There are several very well known providers of BIOS code, and PCs are built to conform to a very rigid set of rules, which means that PCs are very similar to each other in hardware terms. The equivalent software in an Alpha-based system from Digital (and even in a VAX-based system) is the console, and within Digital it is known as the SRM console, since its interface is described in the System Reference Manual.
So, just why did Linux need the SRM console? Firstly, Linux needs to be loaded from some media, and the SRM contains device drivers to do just that. Secondly, Linux needed the Digital Unix PALcode. PALcode can be thought of as a tiny software layer that tailors the chip to a particular operating system. It runs in a special mode (PALmode) and has certain restrictions, but it uses the standard Alpha instruction set. In this way, the Alpha chip can run such diverse operating systems as Windows NT, Open VMS, Digital Unix and, of course, Linux. Finally, Jim was using the SRM call backs in his prototype device drivers. From Linux's point of view, though, the SRM Console does too much. It contains call back procedures that allow the running operating system to write messages to the console or to write environment variables and so on. Linux makes no use of these functions; in fact, the only part of the SRM console needed, once it loads Linux, is the PALcode.
I volunteered to write a loader that would be small and would do only those things that Linux needed. Like all good projects, my one-man effort had some straightforward project goals. First and foremost, the software would be under free licence, built and freely distributed as part of standard Linux distributions. Secondly, Linux drivers should be able to be used within Milo without modification or even re-compilation. Thirdly, it should maximize the amount of memory available to Linux. Little did I realise quite what I was letting myself in for.
Milo contains the following functional pieces:
Linux Kernel and pseudo-Kernel
Linux Kernel interface code
User interface code
Finding Digital Unix PALcode was no problem. Back in 1992 when Digital announced the Alpha processor, it also announced that it was moving into the merchant chip market. I joined a small group in the UK to provide engineering effort in Europe to further these aims. We are a small offshoot of the main group, which is based in the silicon factory in Hudson, Mass. We build evaluation boards for the Alpha processors and PCI peripheral chips. These systems included a very low level Evaluation Board Debug Monitor that uses Digital Unix PALcode. The sources for the Evaluation Board Debug Monitor and the PALcode are under free licence.
Although this PALcode is fully compliant with the interface described in the Alpha Architecture Manual, there are some differences between it and the SRM console's PALcode. One of the differences between Alpha based systems is the way interrupts are handled outside of the processor itself. There are a limited number of interrupt signals into the CPU itself, and the number varies from CPU to CPU, but there are typically three: timer, I/O and non-maskable interrupt.
The way in which real device interrupts are mapped onto CPU interrupts is system-specific. Most current Alpha systems include an ISA bus, whose interrupts are routed through a pair of 8259s in the same way as on x86 PCs. The SRM console PALcode handles these differences and interprets the interrupt, passing it to the OS's interrupt handling code as an “SCB offset” (described by Jim Paradis in last month's Kernel Korner). The PALcode used in Milo does not do this interpretation, so the OS's interrupt handling code must do it instead. Particularly with PCI devices, there must be code in the PCI BIOS code and within the interrupt handler that understands how interrupts are routed in the system. One side effect of this is that when Linux has been loaded by Milo, the interrupt handler can handle more than one device's interrupt each time it is called.
One interesting and useful feature of the example PALcode I adopted from our Alpha evaluation boards is that it allows the Evaluation Board Debug Monitor to run in 1-to-1 physical addressing mode. Bit 0 of the virtual page table base register turns this feature on and off. When translation buffer misses occur, the PALcode builds a new page table entry and inserts it into the cache. Pretty much the first thing that Milo does when it is loaded is to swap to this PALcode in physical address mode. The last thing that the Mini loader does is to swap to this PALcode again, this time passing final control to the Linux kernel.
Milo must turn virtual memory mapping on as it passes control to the kernel, because Linux expects that a control structure called the Hardware Restart Parameter Block (HWRPB) is at the right virtual address. Amongst other things, this describes the type of system and how much memory is free, together with where the memory is. As Linux was first loaded via the SRM console, it naturally used the interface provided by the SRM, which was the HWRPB, as described in the Alpha Architecture Manual. I could see no reason to change this interface: there are enough interfaces in the world, so why invent yet another one?
In order for Milo to set up the memory mapping correctly, it must itself have a good idea of what memory is available and what it is being used for. It finds the amount of memory available because after the PAL reset code has been executed, the size of memory is put into the impure area, a data structure shared between the PALcode and the console or Evaluation Board Debug Monitor. Milo keeps a memory map describing what each page in the system is being used for. While device drivers are running, they allocate temporary memory and use it. Just before control is passed to Linux, Milo must build a correct memory cluster description in the HWRPB, and the memory map is used to do this. Pages are marked as “free”, “allocated” or “temporarily allocated” in the memory map. When Milo builds the memory cluster descriptions in the HWRPB, it treats all temporarily allocated memory as free, since they will be free once Linux starts to run. In this way, the only memory that is marked as allocated is the memory containing the PALcode (8 pages), the memory for the HWRPB (1 page) and the memory for the level 2 and level 3 page tables (2 pages): 11 pages in total. I think I've succeeded in my goal of maximizing the amount of memory available to Linux.
Practical books for the most technical people on the planet. Newly available books include:
- Agile Product Development by Ted Schmidt
- Improve Business Processes with an Enterprise Job Scheduler by Mike Diehl
- Finding Your Way: Mapping Your Network to Improve Manageability by Bill Childers
- DIY Commerce Site by Reven Lerner
Plus many more.
- Unikernels, Docker, and Why You Should Care
- Happy GPL Birthday VLC!
- What's New in 3D Printing, Part III: the Software
- Handheld Emulation: Achievement Unlocked!
- What's New in 3D Printing, Part II: the Hardware
- Giving Silos Their Due
- Non-Linux FOSS: Snk
- Controversy at the Linux Foundation
- Don't Burn Your Android Yet
- February 2016 Issue of Linux Journal