AIPS and Linux: A Historical Reminiscence
It was a dark and stormy night. Well, maybe not—but there was plenty of inclement weather on a late fall day in 1993, in central Virginia. I was waiting with eager anticipation for the arrival of a third-year college student from Virginia Tech, who had some precious cargo in his possession: a working version of our flagship application, AIPS, that he had ported to a brand-new and relatively obscure operating system. He also brought something that would revolutionize the basis for computing at my workplace: a QIC-60 cartridge containing a very early, well-hacked and customized version of the SLS distribution of Linux. I believe it was based on kernel 0.99.12, although it could have been 0.99.11.
I work at the Headquarters of the National Radio Astronomy Observatory (NRAO) in Charlottesville, Virginia. Our mission is to provide world-class facilities for astronomers to observe the universe through radio waves. Despite what most people think, we don't listen to anything. Instead, we gather signals, grind them through some impressive hardware and software and end up with pictures of what the sky would look like if we could see in the radio part of the electromagnetic spectrum.
On that particular day in 1993, my main duty was systems programming and world-wide installation support for AIPS, the Astronomical Image Processing System. This is NRAO's main application for processing the signals from our telescopes. It is a combination of command-line interpreter, graphical image display and a large (300+) set of programs or “tasks” that perform more-specialized functions. These functions range from simple bookkeeping tasks to serious number-crunching algorithms such as deconvolution, maximum entropy, Fourier transforms and more.
Established in the waning years of the 1950s, the National Radio Astronomy Observatory is a facility that now comprises telescopes at several widely scattered sites across the United States. Headquartered in Charlottesville, the main instruments it operates are the Very Large Array (VLA), located 50 miles west of Socorro, New Mexico; the Green Bank Telescope (GBT, due to be commissioned in August 2000) in West Virginia, and the Very Long Baseline Array (VLBA) which has ten large telescopes in locations ranging from Hawaii to New Hampshire, and Washington State to the U.S. Virgin Islands. Visitor centers are located in Green Bank and at the VLA. Anyone who has seen either the movie Contact or 2010 has seen the VLA; both those movies were shot partially on location out on the dusty plains of San Augustin at the VLA site. And no, there isn't a canyon next to the VLA; that was artistic license on the part of the Contact directors! The canyon is actually Canyon de Chelly in neighboring Arizona.
In addition, the NRAO is working with several European partners in a project to create another array, this time for higher-frequency (millimeter wavelength) radio waves. This venture will comprise 64 moderate-size movable telescopes to be located on a 15,000+ foot high plain called Atacama in the remote Chilean Andes mountains. Currently in the design phase, this Atacama Large Millimeter-wave Array (ALMA) promises to open more new frontiers in astronomy.
As can be seen, we use arrays of radio telescopes in most of our instruments. Working together, they can give a much better picture than if they act alone. The technique used to correlate the signals from all the telescopes in an array is called Aperture Synthesis and is used at all our current arrays. This technique is the single most important raison d'etre for AIPS.
Without going into excessive technical detail, here's how aperture synthesis works. Several radio telescopes (antennae) are used, and the signals from each of them are captured, sent via wire or microwave “waveguide”, digitized and fed into a special-purpose computer known as the “correlator”. This one-of-a-kind computer takes the signals from each antenna and correlates them with all other signals. The result is a series of signals called “baselines”, as they represent the correlation of the signal from the ends of a baseline joining the two antennae. For a set of n antennae, you get nx(n-1)/2 baselines. If n is 4, for example, you get (4x3)/2 or 6 baselines (see Figure 3). You don't correlate an antenna with itself, and you don't count antenna.1-antenna.2 and antenna.2-antenna.1 baselines as separate.
When the signals are correlated like this, you end up with a set of “visibilities” that is almost like a Fourier transform of the signals as they arrive at each antenna. Basically, each baseline gives a point on the “U-V” plane, and gridding, interpolating and transforming the result can produce an actual image. The truly cool thing is that this image has the same resolution as if it were taken with a telescope the same width as the whole array. So the VLA in its “A” configuration, with over 15 miles separating the farthest-flung telescopes, is the equivalent of a 15-mile-diameter telescope. It gets better; with the VLBA, we've got the better part of the Earth's diameter between the antennae in Hawaii and St. Croix. As many of you may know, in astronomy it's diameter that counts when you want to “zoom in”. Instead of measuring angular distances in seconds of arc (the moon appears to be about 30 minutes of arc, or 1800 seconds of arc across), we measure things in milli-arc seconds with the VLBA.
Practical Task Scheduling Deployment
July 20, 2016 12:00 pm CDT
One of the best things about the UNIX environment (aside from being stable and efficient) is the vast array of software tools available to help you do your job. Traditionally, a UNIX tool does only one thing, but does that one thing very well. For example, grep is very easy to use and can search vast amounts of data quickly. The find tool can find a particular file or files based on all kinds of criteria. It's pretty easy to string these tools together to build even more powerful tools, such as a tool that finds all of the .log files in the /home directory and searches each one for a particular entry. This erector-set mentality allows UNIX system administrators to seem to always have the right tool for the job.
Cron traditionally has been considered another such a tool for job scheduling, but is it enough? This webinar considers that very question. The first part builds on a previous Geek Guide, Beyond Cron, and briefly describes how to know when it might be time to consider upgrading your job scheduling infrastructure. The second part presents an actual planning and implementation framework.
Join Linux Journal's Mike Diehl and Pat Cameron of Help Systems.
Free to Linux Journal readers.Register Now!
- Paranoid Penguin - Building a Secure Squid Web Proxy, Part IV
- SUSE LLC's SUSE Manager
- Google's SwiftShader Released
- Managing Linux Using Puppet
- Murat Yener and Onur Dundar's Expert Android Studio (Wrox)
- My +1 Sword of Productivity
- Non-Linux FOSS: Caffeine!
- SuperTuxKart 0.9.2 Released
- Parsing an RSS News Feed with a Bash Script
- Doing for User Space What We Did for Kernel Space
With all the industry talk about the benefits of Linux on Power and all the performance advantages offered by its open architecture, you may be considering a move in that direction. If you are thinking about analytics, big data and cloud computing, you would be right to evaluate Power. The idea of using commodity x86 hardware and replacing it every three years is an outdated cost model. It doesn’t consider the total cost of ownership, and it doesn’t consider the advantage of real processing power, high-availability and multithreading like a demon.
This ebook takes a look at some of the practical applications of the Linux on Power platform and ways you might bring all the performance power of this open architecture to bear for your organization. There are no smoke and mirrors here—just hard, cold, empirical evidence provided by independent sources. I also consider some innovative ways Linux on Power will be used in the future.Get the Guide