# Introduction to Lisp-Stat

Although I installed Linux on a 80 Meg
partition on my Gateway 33Mhz PC over a year ago, I really did not
make serious use of Linux for my scientific work, mostly because I
lacked disk space. Recently, I bought a new 1-Gig drive and that
excuse went away. So I decided to install
**Lisp-Stat**, a program that I use
most for my statistical Computing.

Written by Luke Tierney at the University of Minnesota, Lisp-Stat is a powerful, interactive, object-oriented statistical computing environment based on the the Xlisp dialect of Lisp. It runs on under Microsoft Windows, Macs, and Unix based X11 systems almost uniformly. It has good graphical facilities for both static and dynamic graphics along with functions for common statistical computations.

Furthermore, using the foreign function interface, one can call C and Fortran programs from within Lisp-Stat. A byte-code compiler is available to speed up your programs once you have debugged them. Of course, one needn't use Lisp-Stat for statistical computing alone; I routinely use for all kinds of things: as a calculator, for figuring out grades of my students, as an engine for hypertext illustrations as well as matrix manipulations.

In this article, I will introduce you to some of the
capabilities of Lisp-Stat. Although I use Lisp in this article, I
will not get into the details of Lisp programming unless it
impinges on our discussion. If you are new to Lisp, you might want
to read article on Scheme by Robert Sanders, *Linux Journal*, March 1995, as Lisp and Scheme are closely
related. Some of the comments I make apply actually to Lisp, but it
serves no useful purpose to delineate what is the Lisp and what is
the Stat part. You need not know Lisp to use any of the examples or
to follow the article. If you get seriously interested in Lisp-Stat
you should probably get a copy of Tierney's book titled
**Lisp-Stat**, ISBN 0-471-50916-7,
published by John Wiley. Besides being the canonical reference for
Lisp-Stat, it provides a quick and practical introduction to
Lisp.

Assuming that you have installed Lisp-Stat successfully, just
type **xlispstat** to invoke the program. To quit
the program, just type **(exit)**. Figure 1 shows a
simple session. Case does not matter and the
**>** you see in the figure is Lisp-Stat's
prompt. The data I have used is the number of requests a WWW server
honored during each of the *24* hours in a day.
The **def** macro binds a variable name
**requests** to the list of values.

As the example shows, calculation of summary statistics like
the mean and standard deviation are trivial. Since Lisp-Stat is
based on Lisp, you have all the power of Lisp for data
manipulation. A rich set of data types is available including
vectors, sequences, strings, matrices. In figure 1 the variable
*A* is defined to be a 3x3 matrix and
*B* is a list of three numbers. The example
solves *Ax=b* for *x* by
computing *A<+>-1<+>b* yielding the
solution *[2, -6, 1]*. Note that
*b* is a list while
*A<+>-1<+>* is a matrix, yet the
Lisp interpreter takes care of the types and in effect computes the
product of a matrix and a vector.

Many of Lisp-Stat's functions operate on sequences which may
be lists or vectors and they are *vectorized*,
meaning that these functions can be applied to arguments that are
lists and the result is a list of the results of applying the
function to each element of the list. Some other functions are
*vector reducing*, meaning that they can be
applied to a list of arguments but they return a single number. In
figure 1, the function **mean** is an example of a
vector-reducing function; it treated the list of lists as a single
long list and returned the mean of the long list. On the other
hand, the function **normal-cdf** is a vectorized
function and invoking it on a list of three numbers produces a list
of three answers. Of course, if we do wish mean to behave in a
vectorized fashion, the statement **(mapcar #'mean (list
(normal-rand 10) (normal-rand 20)))** will do it.

**Figure 2: A
histogram and a Line Plot**

A picture is worth a thousand words, particularly in statistics. Lisp-Stat boasts excellent graphical tools. The graphical system is based on an object-oriented paradigm. Functions that create graphical windows or plots return an object as the result. The returned object is just another data type much like a number or a list and it can be used in appropriate computations.

Commonly used graphical functions are
**histogram** for constructing histograms,
**plot-points** for plotting
*(x,y)* pairs, **plot-lines** for
joining *(x,y)* pairs by means of lines,
**plot-function** for plotting a function of one
variable, **spin-function** for plotting a function
of two variables, and **spin-plot** for 3-d
plots.

All the **spin** functions provide controls
for yawing, pitching and rolling in the graph they create. Figure 2
shows a plot of the number of requests versus each of the
*24* hours. The plots were produced using the
following lines of code.

(histogram requests) (def time (iseq 24)) (plot-lines time requests) (send * :add-points time requests) (send ** :point-symbol (iseq 24) 'diamond)

Just drawing the lines alone is less than satisfactory since
the exact location of the points is lost. So, after constructing
the plot, we send a “message” to the plot using the
**send** function asking the object to add-points to
the graph resulting in the graph shown. The ***** in
the send function refers to the result of the previous command,
i.e., the plot object. The ****** refers to the
result of the command before the previous one. In the example, I
have asked that the plotting symbol be a diamond instead of the
default circle. The user has a choice of quite a few plotting
symbols.

In each plot there is a menu button that has further useful
options. One can select or deselect points with the mouse,
highlight certain points, save the plot as a postscript file etc. I
will only discuss a single feature, that of
*linking*. Linked plots are a way of sharing
information between plots. Consider for example, figure 2, where we
have a plot of **requests** versus
**time** as well as a histogram of
**requests**.

If you enable linking by choosing the **Link
View** item in the menu in each plot, selecting a vertical
bar in the histogram by dragging the mouse with the button pressed
causes the corresponding points in the line plot to be highlighted.
You might have to peer at the figure to see that the point where
the highest peak occurs is highlighted since it corresponds to the
highlighted histogram bar. Linking is extremely useful in viewing
multidimensional data since one can get a better idea of how the
same group of points can be projected in different views.

Online documentation for Lisp-Stat is available via the
functions **help**, **help*** and
**apropos**. For help on the **mean**
function, type **(help 'mean)**. The use of the
quote is essential, otherwise the interpreter would assume that
mean is a variable and try to evaluate it. However, in many
situations, one does not know what the function is named.

For example, is the function that multiples two matrices
**mat-mult** or **matrix-multiply**?
Typing **(apropos 'mult)** will print a list of all
symbols that have the word “mult” in them. This might help you
narrow down the search. On the other hand, if you know that the
function you are looking for contains the word
**matrix** in it, **(help* 'matrix)**
will return help on all symbols that contain the word
**matrix**. The help facility as it exists now is
less than optimal and several people are developing a more
elaborate help system.

I usually read the newsgroup sci.stat.math and almost always
there is someone out there who wants to know how to calculate an
*F*-probability or how to generate a normal
random variable. Lisp-Stat has distribution functions and
generators for all of the commonly used distributions. For example
**(normal-cdf 1.645)** will give you the probability
to the left of *1.645* which is about
*0.95*. The statement**(def x
(normal-rand 100))** will define x to be a list of
*100* standard normal variates. Similar
functions exist for Students-*T*, Gamma, Beta,
Chi-squared and *F* distributions as
**(help* 'cdf)** or **(help* 'rand)**
will show.

Lisp-Stat has many functions for input and output. For
dealing with files, I've rarely needed to go beyond using the two
functions **read-data-file** and
**read-data-columns**. The statement
**(read-data-file "foo.dat")** returns the whole
file contents as one long list, while **(read-data-columns
"foo.dat")** returns a list of columns of the file. One can
specify the number of columns in the data file as a second argument
to **read-data-columns**. Otherwise, it guesses the
number of columns based on the first line. The function
**format**, which is similar to C's
**sprintf()**, is a versatile function for formatted
printing.