Scientific Visualization with NCL

Many of my previous articles have looked at software packages that do scientific calculations and generate scientific results. But, columns of numbers are nearly impossible to make sense of—at least, by regular human beings. So what can you do? The answer is visualization. We do massive amounts of processing visually, and the easiest way for us to review information is through some graphical format. Because of the power of graphical representations, several packages have been written by different groups. In this article, I'm looking at NCL (the NCAR Command Language, http://www.ncl.ucar.edu). NCAR is the National Center for Atmospheric Research, where the Computational and Information Systems Laboratory develops NCL. NCL is an interpreted language designed specifically for data analysis and visualization. It supports several different file formats used in scientific computations, and it also provides several data analysis functions built-in.

NCL is available as both source code and precompiled binaries. These are hosted at the Earth System Grid Web site (http://www.earthsystemgrid.org). The first step is to get an account on the Earth System Grid site. Once your registration is complete, you can download either the source code or the binaries for the system that interests you. The binaries are packaged as tarballs, and they have been compiled for both Debian-based and Red Hat-based systems. You can select between 32-bit and 64-bit binaries. Once you have the correct tarball downloaded, you can unpack it in the directory of your choice. The usual place on UNIX systems is /usr/local, but you are free to place it anywhere on your system.

After unpacking, you need to set several environment variables before actually using NCL. The first is the environment variable NCARG_ROOT. You need to set this to the parent directory containing the NCL executables and libraries. For example, if you unpack NCL in /usr/local, you would set NCARG_ROOT=/usr/local. You also need to add the location of the NCL executables to your path. In this example, you would add $NCARG_ROOT/bin or /usr/local/bin to the PATH environment variable. You also need to create a configuration file in your home directory, named .hluresfile. This file will hold configuration options like the default font and the default color table.

Once everything is set up, you quickly can test whether NCL is working correctly by running:

ng4ex gsun01n

This command copies an NCL script file named gsun01n.ncl into your current working directory and runs it through NCL to produce some graphical output.

If you decide to build NCL from source, you need to install a number of extra libraries to handle all the possible input file formats, as well as all the possible output graphics formats. A full set of instructions is available at http://www.ncl.ucar.edu/Download/build_from_src.shtml.

It is strongly advised that you try to use the prebuilt binaries first, before going through all the work involved in building NCL from scratch.

Now that NCL is installed and ready to use, what can you actually do with it? If you enter the command ncl, you will get a prompt where you can enter commands. Entering the following command tells NCL to start recording the commands you issue in your current NCL session:

record "my_script.ncl"

When you are done, type stop record to tell NCL to stop recording your session. At this point, you will have an NCL script that you can reuse later, if you wish.

Once you have your NCL script, you can run it through NCL with the command:

ncl <my_script.ncl

This is only one way to develop an NCL script. Remember, NCL is a full programming language, and you can write your scripts from scratch, once you know enough of the commands that are available.

The first structure to learn is the format of an NCL script. Scripts all start with the command begin and finish with the command end. Everything else happens between those two statements.

In NCL, four general groups of objects are available. The first group is the workstation objects. These objects represent the graphical devices that act as display devices that the graphical functions use to draw on. These could be an X11 window, an NCAR Computer Graphics Metafile or a PostScript file. This second group is data objects. Data objects store the actual information used in your analysis and graphical presentation. The third group is the view objects, which represent the elements of your graphical representation. These might be things like text objects, tick marks or contour plots, among others. The fourth is the group of "other" objects, which includes everything else, such as overlays or annotations. You can create new objects by using the commands:

objectname = create "object_character_name" 
 ↪class_name parent_object
end create

where you create a new object based off an existing parent object.

So, what does a basic graphical display look like? You can generate something simple with the following example:

begin
x11 = create "x11" xWorkstationClass defaultapp
end create
text = create "text" textItemClass x11
end create
draw(text)
frame(x11)
end

Save these commands into a file named sample1.ncl. To run it, you can execute ncl <sample1.ncl. The first three lines create an X11 object to draw on. The next two lines create a text object. You need the draw command to generate the text object, while the frame command instantiates the graphical display on your screen.

What if you want to import data in order to do some processing? NCL can handle many of the file formats used in scientific computational work. As a simple case, let's say you have some code that simply dumps the result as an ASCII representation of the values of interest. You can load this data with the command:

file_data = asciiread("/full/path/to/file", file_size, "float")

This will read in file_size numbers from the file given and import them as a float type. Once you've done this, you can grab subsets using indices and assign them to variables. For example, you could create a new array with:

array1 = new(64, float)

and then assign the first 64 elements from the file with the command:

array1(0:63) = file_data(0:63)

Graphing is a bit more complex. The first step is to create a data object that will store the values to be graphed. For example, you could use something like this:

field1 = create "field1" coordArraysClass defaultapp
   "caYArray": (Y-data)
   "caXArray": (X-data)
end create

where you would replace X-data and Y-data with the actual data arrays you want to use. Once this is done, you actually can do the graphing with:

xy_plot = create "xy_plot" xyPlotClass wks
   "xyCoordData": field1
end create
draw(xy_plot)
frame(wks)

This creates an xy_plot object, based on the data you imported into NCL. Next you need to run the draw command and then call the frame command to instantiate the plot.

If you simply want to view the graph, you can call frame on an X11 display object. Or, you can save the graph to a file by calling frame with a file object, like a PostScript file.

There are many possible gotchas when writing and running NCL scripts. Many of these will lead to inefficient code, mostly due to the fact that NCL is an interpreted language. The first item to look at is whether you are using loops unnecessarily. Whenever possible, you should try to use built-in functions to do any processing, rather than try to do the processing yourself. As an example, say you wanted to multiply two 100x100 arrays. If you did it yourself, you probably would write some looping code like this:

do i = 0,99
   do j = 0,99
      c(i,j) = a(i,j)*b(i,j)
   end do
end do

In NCL, you equivalently could write:

c = a*b

This is not only shorter, it is also much more efficient. Instead of having to go through each iteration of the loop and ending up incurring the cost of evaluating the statements on each iteration of the loop, the multiplication is evaluated as a single statement by NCL and is handed to the underlying libraries that handle matrix multiplication as a single statement. You also should try to make sure that you pull out as many operations from the loop as possible and apply them a single time after the loop operation is finished. Additionally, sometimes the processing you are trying to do is simply too intensive. In those cases, you can tell NCL to load and use external C- or FORTRAN-compiled objects to handle this processing.

This has been only the barest of introductions to NCL. NCL is a full programming language, with loop structures and conditional statements. You have variables, objects and loads of built-in functions. You can check out all of the possibilities by visiting the NCL and Earth System Grid Web sites. You might be surprised at the analysis that is possible.