NetDVD: Building a Network-Attached Peripheral with Linux
I work in the Molecular Imaging group of Siemens Medical Solutions, where my colleagues and I develop and maintain software used to run Positron Emission Tomography (PET) medical scanners. These machines generate large amounts of patient data that our customers must archive for later retrieval and review, so our software allows customers to archive data to tape or Magneto-Optical (MO) disk. Using tape is very slow, so archiving to MO disk is generally preferred. Unfortunately, during the past few years, we've had a lot of problems with the MO hardware sporadically corrupting the MO disks, leading to expensive and tedious data recovery efforts and several replaced MO drives.
For many years, our customers have been asking for the ability to archive their data to DVD. DVD media is much cheaper than either tape or MO, and it can be read by any PC with a DVD drive. This would be useful to many of our customers, because there are free software tools they can use to read our data files on a PC. Because the MO drive vendor appeared unable to resolve our hardware problems, we decided the time was right to implement DVD archiving.
Unfortunately, we immediately hit a snag. The control consoles for our scanners are Sun UltraSPARC workstations running Solaris 2.6, Solaris 7 or Solaris 8. Customer hardware ranges from an Ultra 2 to a Sun Blade 2500. With so many different machines to support, we clearly needed an external SCSI DVD burner. But, we couldn't find a stable source for such a device, and Solaris 2.6 and Solaris 7 have little or no support for DVD burner hardware.
Because DVD burners are easy to get for x86 hardware and have been well supported by Linux for years, we decided the best solution would be to use a small Linux box with a high-quality DVD burner to do the work of reading and writing the DVDs. This would solve our hardware and OS-compatibility problems and make it easier to add support for new media types in the future. The Solaris workstations would communicate with the DVD reader/writer machine over the network, so we named our creation the NetDVD.
Because our business is building large medical scanners—not computers—the first thing we needed was a company to put the devices together for us and help select the hardware. I did Web searches to find appropriate companies and sent out several query letters. Few of the companies I contacted were interested in our project, and most required a guaranteed minimum number of units to work with us. We have about 700 scanners in the field, and it's likely we'll sell NetDVD devices to a few hundred of those, but I didn't have the authority to make any kind of commitment.
Thankfully, MBX Systems was very helpful. Their representative, Ed Jamison, quickly suggested some possible hardware solutions, including one using a mini-ITX motherboard in the C134 case from Casetronic, shown in Figure 1. This case is actually a bit smaller than the external MO drives we've been using, so it looked like the perfect choice. We eagerly ordered one with a 1GHz processor, 256MB of RAM, a 40GB hard drive and a laptop-size DVD burner.
Once we had the prototype, we had to decide what Linux distribution we would put on it. My fellow developer and Linux enthusiast, Dan Duckworth, had just been reading about how great this new Ubuntu distribution was, so we decided to give it a try. We downloaded the Ubuntu 4.10 Warty Warthog CD image, and it installed beautifully. Ubuntu is based on Debian, which I've been using for several years, so I found configuring it very easy to do. It worked so well for us that we never even tried another distribution.
While we were picking out a vendor, hardware and a Linux distribution, I was putting a lot of thought into how client machines would talk to the NetDVD. The client computer must be able to read a DVD, write to a DVD and create a copy of a DVD to provide redundant archive backups. To manage this, I decided to combine a custom TCP/IP protocol with Network File System (NFS).
To use a NetDVD, the client machine connects to it using our custom NetDVD TCP/IP protocol. If the device is already busy serving another client, it will respond with a BUSY error message and drop the connection. This ensures that only one client can control the device at a time. If the device is not busy, it will send a brief message stating the highest version of the NetDVD protocol it understands. At the time of this writing, the only version of the NetDVD protocol is 0, but we may create new versions of the protocol later to add new features.
If the client connects successfully, it immediately sends an initialization command to the NetDVD. This command tells the device both the version of the NetDVD protocol the client will use to communicate with it and what the client believes the current time to be in Universal Coordinated Time. The NetDVD sets its clock to match the client, so the machines will be in agreement on filesystem timestamps.
Also during the initialization command, the NetDVD uses NFS to export an empty directory on its hard drive with read/write permission strictly to the client's IP address. This is a working directory for the client to fill with files it wants to write to DVD. It is on a large disk partition separate from the operating system, so filling it up will not cause the device to misbehave.
To read a DVD in the NetDVD's drive, the client sends a mount command. This causes the device to mount the media in its drive and export it via NFS strictly to the client's IP address. When the client is finished reading, it can simply unmount it or unmount and eject it using an appropriate command.
To write files to a DVD, the client first mounts the NetDVD's working directory via NFS and fills it with the files and directories it wants to write to the DVD. Once finished, it sends a burn working directory command to the device and specifies whether this is supposed to be a new DVD or data appended to a DVD written previously. The ability to append data was crucial to our use of the device to archive data incrementally.
To copy a DVD in the NetDVD device's drive, the client sends a special copy command. The device then copies the directory structure on the media to a special directory on the same partition with the working directory. Once the copy is done, the client may send a burn copy command to the device one or more times to write the copied directory tree to as many DVDs as it likes. This method will not make viewable copies of video DVDs, because the data on those disks must be written in a very specific order that we don't preserve.
Fast/Flexible Linux OS Recovery
On Demand Now
In this live one-hour webinar, learn how to enhance your existing backup strategies for complete disaster recovery preparedness using Storix System Backup Administrator (SBAdmin), a highly flexible full-system recovery solution for UNIX and Linux systems.
Join Linux Journal's Shawn Powers and David Huffman, President/CEO, Storix, Inc.
Free to Linux Journal readers.Register Now!
- Tips for Optimizing Linux Memory Usage
- Secure Desktops with Qubes: Introduction
- Working with Command Arguments
- Download "Linux Management with Red Hat Satellite: Measuring Business Impact and ROI"
- Fancy Tricks for Changing Numeric Base
- Secure Desktops with Qubes: Installation
- CentOS 6.8 Released
- Linux Mint 18
- The Italian Army Switches to LibreOffice
- Petros Koutoupis' RapidDisk
Until recently, IBM’s Power Platform was looked upon as being the system that hosted IBM’s flavor of UNIX and proprietary operating system called IBM i. These servers often are found in medium-size businesses running ERP, CRM and financials for on-premise customers. By enabling the Power platform to run the Linux OS, IBM now has positioned Power to be the platform of choice for those already running Linux that are facing scalability issues, especially customers looking at analytics, big data or cloud computing.
￼Running Linux on IBM’s Power hardware offers some obvious benefits, including improved processing speed and memory bandwidth, inherent security, and simpler deployment and management. But if you look beyond the impressive architecture, you’ll also find an open ecosystem that has given rise to a strong, innovative community, as well as an inventory of system and network management applications that really help leverage the benefits offered by running Linux on Power.Get the Guide