Autonomous Vehicles

The goal of the ARGO Project is to develop active safety systems for vehicles of the future. These systems are able to understand the environmental conditions and, in case of sudden danger, they can warn the driver or even take control of the vehicle. Furthermore, these systems have the capability to fully drive the vehicle without human intervention (Automatic Driving).

Figure 1. The ARGO Prototype Vehicle

ARGO (see Figure 1), the prototype vehicle developed at the Università di Parma, Italy, was demonstrated to the public and the scientific community in June 1998, when it drove autonomously for more than 2000km on the Italian highway network.

The entire real-time data acquisition and processing is performed on a single Pentium MMX-based PC running Linux.

A large number of problems (not just those related to traffic and safety) could be solved by using automatically driven vehicles.

Beside the obvious advantages of increasing safety and reducing road accidents, thereby saving lives, the possibility of having vehicles moving in much closer proximity than they do today would produce an increase in road capacity. A more intelligent modulation of each vehicle's speed would also result in an appreciable reduction in fuel consumption. In other words, automatic vehicle driving has the potential to achieve optimal use of current transportation infrastructures, improve mobility and minimize risks, travel times and energy consumption. Moreover, commercial and industrial vehicles which repeatedly move along a given path would benefit from a stronger control of their routes and would require fewer personnel to manage their moves.

Unfortunately, an automatic vehicle, more than in other applications, needs real-time response, thus requiring smart algorithms and powerful computing engines. At the same time, as far as a commercial product is concerned, the cost of the system must be kept low. Initially, the underlying technology, such as head-up displays, infrared cameras, radars and sonars are derived from expensive military applications. Thanks to increased interest in these applications and the progress of industrial production, today's technology offers sensors, processing systems and output devices at very competitive prices.

This project shows how a very low-cost solution (regarding both sensors and processing engines) was successfully adapted to drive an “intelligent” vehicle in real-world conditions. This article describes the underlying architecture, which is based on a standard Pentium MMX 200 MHz and the Linux operating system, and discusses the main advantages and problems encountered during the whole project.

The project was founded by the Italian National Research Council (CNR), and the MilleMiglia in Automatico tour was sponsored by TIM, Telecom Italia Mobile, which provided GSM apparatus and cellular connectivity throughout the whole journey.

The heart of the computing architecture installed on ARGO (shown in Figures 2 and 3) is based on a single Pentium MMX processor (200MHz, 32MB of RAM). The PC was equipped with some additional boards for image acquisition, image visualization, acoustic warnings and data I/O from specific devices.

Figure 2. Internal View of the ARGO Vehicle

Figure 3. The Computing Architecture and Equipment

GOLD (Generic Obstacle and Lane Detection), the software that drives the ARGO vehicle, has been designed to be as portable as possible; therefore, it is independent of specific Linux distributions/libraries and can be compiled against either libc or glibc. Moreover, because of the high stability of the first kernel (2.0.18) we installed, the current distribution used on ARGO is still a Debian 1.2.

The following is a description of the ARGO adapters/software libraries as well as their support for Linux.

Acquisition device: since ARGO relies on a vision-based processing engine, the acquisition device is the most important piece of hardware. GOLD needs a frame grabber capable of simultaneously grabbing two grey-level images from two different cameras. Thanks to the availability of specific mailing lists about Linux and Vision (http://atlantek.com.au/USERS/wes/linux/frame.html), we discovered that only the Matrox Meteor RGB frame grabber had this feature among the Linux-supported hardware. The Matrox Meteor module is not yet video4linux compliant. Nevertheless, the availability of many examples as well as their source code permitted us to rapidly interact with this hardware.

Data I/O: while the frame grabber could be devised as the main input device, autonomous driving also requires an output device for maneuvering the steering wheel. A National Instruments LabPC+ ISA adapter has been installed in the ARGO PC; it provides a number of multi-functional analog, digital and timing I/O ports. Therefore, it is used for driving the steering wheel actuator (a stepping motor), determining the vehicle's speed (through a Hall-effect speedometer) and inputting user commands from the control panel as in Figure 2. The LabPC+ usage is quite simple, since the Linux module supplies a device and simple ioctls for each different I/O port.

Acoustic warnings: acoustical warnings are fed to the driver. A simple interface for a standard sound card has been developed using the OSS free sound module.

Figure 4. A Screenshot of the Output from the Onboard Monitor

Image visualization: the result of the computation is also fed to passengers through a color 6-inch monitor installed onboard the ARGO vehicle (see Figure 4). Development of the software requires the capability to both inspect intermediate results and input debug commands. For these purposes, two interfaces have been developed, one VGA-based and one X11-based. In the first case, the SVGA library is used for showing the (intermediate) results and the ncurses library is used to input user commands. In the second case, the xform widget library is also used (see Figure 5).

Figure 5. The X11-based Control Panel

Programming facilities: the MMX capabilities of the Pentium processor were exploited in order to boost processing speed. A few portions of the GOLD code were directly rewritten using MMX assembly language and compiled using the Netwide Assembler (NASM), a general-purpose x86 assembler that supports Pentium, P6 and MMX opcodes.

Internet connection: during the vehicle's demonstrations, live video shots of the driving cabin are broadcast to the Internet. To accomplish this, a link to the Internet was established. This type of link implies the use of mobile telecommunications facilities, such as GSM or satellite modems and places the following constraints:

In order to increase the link's throughput, two GSM modems are used simultaneously. In fact, the Linux kernel provides a specific method of making multiple serial links behave as a single faster connection: the EQualize Load balancer (EQL). The underlying idea is to split network traffic across the serial lines. In addition, the EQL also supports links that feature different throughputs and protocols.

In order to not excessively burden the main processing engine, another cheap Linux box (a Compaq laptop) was installed on the vehicle and was equipped with a parallel port Quickcam Color camera (supported by Linux) and two GSM modems able to work up to 9600Kbps using the MNP10-EC protocol. A custom application is used to grab images from the Quickcam, convert them to the JPEG format and send them to our web server at http://MilleMiglia.ce.unipr.it/ (a third Linux machine) through the two GSM modems exploiting EQL. As soon as these images are received, a graphical timestamp is superimposed onto their lower portion and they are made available on the Internet. A simple script running continuously in the background is used to restore the two connections if, for any reason (tunnels, GSM uncovered areas, etc.), they are lost.