MYDATA's Industrial Robots
When I joined MYDATA, I had been using Linux for years. I thought that programming Unix on PC hardware would be comparable to running Linux. MYDATA was then using Venix 3.2, an “industrial strength real-time operating system” from VentureCom based on System V, release 3.2. After a short time of programming Venix I was very disappointed. None of the tools that I usually relied on were available: no emacs, no bash, not even a strerror function in the C library.
In the summer of 1995, I began lobbying for a more modern operating system. There were two alternatives: Venix 4.2 (supported mostly by old timers), and Linux 1.2.8 (supported by a few young programmers). A change to either of these operating systems meant a massive porting effort.
After a lot of nagging I got permission to test the interrupt response time for both operating systems under different load conditions. The test was done with a special plug-in card with output ports and interrupt generation connected to an oscilloscope.
Everyone expected this test to kill Linux as an alternative. In Venix I had a hard real-time process, and in Linux I used the nice levels to create a high priority process. The surprising result was that the average interrupt response time was three times faster on Linux—and the worst case interrupt response time was ten times faster on Linux.
Despite the great test results, there was not much interest in Linux. A few programmers wanted to use Linux, but the marketing department had invested a lot in Venix as the “industrial strength real time OS”. There was also strong resistance from people who were unwilling to learn a new operating system.
Then, in December 1995, circumstances changed. 3Com could no longer deliver the 3c503 Ethernet card that Venix 3.2 used. MYDATA managed to find an old stock of WD8003 cards with device drivers for Venix, and I got permission to start a one man project doing a pilot port of the system to Linux.
An industrial robot such as the MYDATA pick-and-place machine has a lot of special hardware. My first job was to rewrite all the device drivers for Linux, which was interesting, produced no surprises—and was completed on schedule. Of course, the fact that I had source code for the Venix device drivers was a distinct advantage.
In early 1996 I learned that Markus Kuhn (a German student) had added POSIX real-time scheduling to Linux 1.3. This was a gift from heaven—hard real-time priorities and memory locking in Linux.
During the spring I worked on porting the application programs. This work proved to be a lot more difficult than I had expected. I would get a program to compile, just to have it immediately crash when running. After long hours of debugging I found an uninitialized variable. Due to different memory layouts, the program just happened to work in Venix. With the bug fixed I would run the program again, and again it would crash for similar reasons.
Another cause of great grief was using C++ objects in file scope and in global scope. If you declare two C++ variable in file scope, the constructors for those variables will be run before main is started. However, if the variables are declared in different files, the order in which the constructors are run will be defined by the implementation. It turned out that the Venix linker ordered the constructors in alphabetical order of object file names, whereas the Linux linker ordered them in the sequence the object files were added to the library using ar (archive).
The result was that suddenly we had a lot of code which contained uninitialized variables. After fighting with the code to solve these problems I am firmly convinced that both file scope and global variables should be banned from C++.
As spring grew into summer I continued in this manner with welcome help from another programmer, Carl Martinsson. Finally, in June 1996, our joint effort was rewarded. We had a MYDATA pick-and-place machine running Linux and actually mounting components on a PCB.
The main lesson I learned from this porting effort is that moving non-portable Unix programs from one Unix-like OS to another is much harder than one might think. Also, rewriting device drivers for Linux is not that hard.
During the work on the port to Linux, the software team developed a new version of the software (with new features) for Venix. In October 1996 we finally received the decision that the next release of the software was to run under Linux.
With the help of our code-management system from Perforce (http://www.perforce.com/), we managed to merge the newly developed features for the Venix system with the Linux system. Of course, this resulted in another round of debugging.
Now, in early 1997, the Linux version of our software is the main development environment and is fully functional. Support hardware currently being developed needs to be completed before we can start the verification tests and ship the software to beta test sites. The release of the Linux-based system to customers worldwide is scheduled for the third quarter of 1997.
The MYDATA pick-and-place software consists of 8,000 files and 150MB of source code. MYDATA is shipping roughly one pick-and-place machine a day at $150,000 each. I think this is the first project to utilize Linux as a real-time platform on this scale.