Introduction to Lisp-Stat

If you do heavy-duty statistical computing and have been looking for a powerful statistical package that runs under Linux, Lisp-Stat may be just what you need.
Programming in Lisp-Stat

Lisp-Stat's graphical system and regression models are implemented using a prototype-based object system. This is different from the class-based object system used by languages like C++ or the approach used by Common Lisp Object System (CLOS). Briefly speaking, there is a root prototype object from which instances of all other objects are created. Objects can have slots to hold information and they respond to messages which are dispatched to the object using the send function. Messages are typically keywords, words that begin with a colon–:add-points in figure 2 is an example.

The code that actually implements the action is called a method for the message. The macros defproto and defmeth make the process of constructing objects and writing methods easier. Lisp-Stat would be less interesting if all it provided were objects for building statistical models. The windowing system provides objects for building user interfaces like menus, dialogs, slider controls etc. So one can construct nice dialogs to go with the computations.

Figure 3: A 2-D Plot with a Least Squares Line Superimposed

A Simple Animation

Figure 3 shows an example of dynamic animation using a slider dialog. The function sin2pi x/n is plotted. The slider allows the user to see the plot change as n is changed. The code to perform this is below.

(setf n 1)
(defun f (x) (sin (/ (* 2 pi x) n)))
(def sine-plot (plot-function #'f -5 5))
(defun change-n (x)
  (setf n x)
  (send sine-plot :clear :draw nil)
  (send sine-plot :add-function #'f -5 5))
(sequence-slider-dialog (iseq 1 20) :action #'change-n)

The function sequence-slider-dialog creates a slider. Initially, the global variable n is 1. Every time the user moves the slider-stop using the mouse, the function change-n gets called with the value of n corresponding to the slider-stop. In our example, n can be any integer from 1 to 20. The function change-n sets the value of n and redraws the plot.

Figure 4: A 2-D plot with a Least Squares Line Superimposed

An object-oriented Programming Example

In order to keep the discussion tolerable, I chose a simple example that is probably not too useful. For serious programming, one needs to know about the built-in prototypes and functions of Lisp-Stat discussed in Tierney's book. I shall introduce what I need as I go along.

We will create an object that accepts a list of (x,y) values and draws a plot with the least-squares line superimposed on it. We will also require that the equation of the least-squares line be displayed in the plot. We begin by defining a new prototype. It is only natural that our prototype be a descendent of the built-in prototype scatterplot-proto which “knows” all about drawing 2D plots.

(defproto least-squares-plot-proto '(intercept slope) ()
          scatterplot-proto)

Notice that our prototype has two slots for holding the intercept and the slope of the least squares line. We will need to access the values in these slots later, so it is best to define two simple methods using the defmeth macro that return the slot values.

(defmeth least-squares-plot-proto :slope ()
 "Returns the slope of the least squares line."
  (slot-value 'slope))
(defmeth least-squares-plot-proto :intercept ()
 "Returns the intercept of the least squares line."
  (slot-value 'intercept))

We have provided a documentation string for the methods; the documentation can be retrieved by means of a command such as (send least-squares-plot-proto :help :slope).

In order to use our prototype, we must define a :isnew method that initializes an instance of the prototype. Our :isnew method must calculate the least-squares line and store the slope and intercept. It should exploit its lineage as a descendant of scatterplot-proto by invoking the inherited methods to do the plotting tasks. Some space must be created in the margin to display the equation for the least-squares line.

Finally, the x,y points must be plotted, the axes labeled, and the window redrawn to reflect the changes. Here is the method.

(defmeth least-squares-plot-proto :isnew (x y &key (title "LS Plot"))
  (let* ((m (regression-model x y :print nil))
         (beta (send m :coef-estimates)))
    (setf (slot-value 'intercept) (select beta 0))
    (setf (slot-value 'slope) (select beta 1)))
  (call-next-method 2 :title title)
  (send self :margin 0 (+ (send self :text-ascent)
                          (send self :text-descent)) 0 0)
  (send self :add-points x y)
  (send self :variable-label 0 "X")
  (send self :variable-label 1 "Y")
  (send self :redraw))

We have used the regression-model function to compute the least-squares line. The call-next-method function calls the :isnew inherited method of scatterplot-proto–this is what actually creates a plot-window. The argument 2 just refers to the number of variables that will be plotted. At this point, the plot-window is actually blank. Using information about the font in use, a margin area is created. Then the points are plotted. In the body of a method the variable self is bound to the object receiving the message. The method concludes by giving some meaningful names to the variables and redrawing the window.

All the above code will do is plot the points. How can we ensure that least-squares line and its equation are also displayed? We use the fact that any window is actually drawn using a :redraw message. By writing a new :redraw message, we can ensure the results we want. In actuality, the :redraw message itself is executed via three other messages :redraw-background, :redraw-content and :redraw-overlays. We really only need to write a :redraw-content method since only the content of the plot is affected. So here we go.

(defmeth least-squares-plot-proto :redraw-content ()
  (call-next-method)  ; Let the scatterplot do its things.
  (send self :adjust-to-data :draw nil) ; make sure scale is ok.
  (let* ((limits (send self :range 0))
         (intercept (send self :intercept))
         (slope (send self :slope))
         (info-str (format nil "y = ~5,3f + ~5,3f x" intercept slope)))
    (send self :draw-string info-str
          10 (+ (send self :text-ascent) (send self :text-descent)))
          ; Display the equation in the margin.
    (send self :add-function ; Draw the LS line.
          #'(lambda (x) (+ intercept (* slope x)))
          (car limits)
          (cadr limits) :draw nil)))

Notice that the keyword argument :draw is nil to avoid infinite loops in the redrawing process. If :draw is not nil, the :redraw method gets invoked again. The line is actually drawn using the :add-function method of scatterplot-proto. We need not worry about drawing the points since that is the responsibility of scatterplot-proto once we have added the points in the :isnew method.

Figure 4 shows the results of using this code with the following program.

(def x (normal-rand 20))
(def y (+ 5 (* 2 x) (normal-rand 20)))
(def m (send least-squares-plot-proto :new x y))
______________________

White Paper
Linux Management with Red Hat Satellite: Measuring Business Impact and ROI

Linux has become a key foundation for supporting today's rapidly growing IT environments. Linux is being used to deploy business applications and databases, trading on its reputation as a low-cost operating environment. For many IT organizations, Linux is a mainstay for deploying Web servers and has evolved from handling basic file, print, and utility workloads to running mission-critical applications and databases, physically, virtually, and in the cloud. As Linux grows in importance in terms of value to the business, managing Linux environments to high standards of service quality — availability, security, and performance — becomes an essential requirement for business success.

Learn More

Sponsored by Red Hat

White Paper
Private PaaS for the Agile Enterprise

If you already use virtualized infrastructure, you are well on your way to leveraging the power of the cloud. Virtualization offers the promise of limitless resources, but how do you manage that scalability when your DevOps team doesn’t scale? In today’s hypercompetitive markets, fast results can make a difference between leading the pack vs. obsolescence. Organizations need more benefits from cloud computing than just raw resources. They need agility, flexibility, convenience, ROI, and control.

Stackato private Platform-as-a-Service technology from ActiveState extends your private cloud infrastructure by creating a private PaaS to provide on-demand availability, flexibility, control, and ultimately, faster time-to-market for your enterprise.

Learn More

Sponsored by ActiveState