Tcl/Tk: The Swiss Army Knife of Web Applications

Sometimes the data you need doesn't come from the user. Product catalogs, maps, schedules and more all come from some sort of external data file, whether real databases like Oracle or Informix or something as simple as a delimited ASCII file. No matter what your data needs are, Tcl can help you out.

Flat files are the easiest to deal with. Open a file, read through line by line until you reach the end or find what you are looking for, then close the file. In Tcl, reading in a file line by line looks like this:

set f [open foo.txt r]
while {[gets $f stuff] != -1} {
  # Do something with the line
  # of data
  (`stuff')
}
close $f

Just as in Perl or C, you create a file handle from which all subsequent operations work. The gets command grabs one line at a time from a file and stores it to a variable. If the return value from gets is 1, you've reached the end of the file. So what do you do with the data once you have it? For the most part, you're going to become fast friends with the split and lindex commands.

split breaks up a string, either character-by-character or at every occurrence of specified characters and returns a new list of the elements. If you want to access specific elements of the list, lindex allows you to specify the list and the element's position and returns that element's value. Note that elements are numbered starting at 0, so an index value of 1 points to the second element in a list.

A bit higher on the effort scale is processing a special database format, such as a dBASE file or some other defined database format. You may be fortunate enough to find existing filters for this kind of file (two different filters exist for dBASE files), but if you need to write your own, Tcl 8.0 handles binary data quite well. Use the read command to grab whatever size byte blocks you want, then use binary scan to quickly break up and format it.

If you're concerned about speed or already have a C routine to parse your external data, Tcl makes it easy to create new Tcl commands encapsulated in loadable libraries. For most functions, it's as easy as cutting and pasting into the library framework provided by Tcl and adding some Tcl-specific commands to create or set variables.

When you get to the top of the database world and are dealing with Oracle or Informix, you're already covered. Tcl extensions have been made for Oracle, Informix and probably others by the time you read this article. Most of them provide access to the SQL layer for the database, but you can also access the lower-level functions of the system. All of them are available on-line, although compiling them sometimes requires access to the commercial libraries shipped with your RDBMS.

One problem with basic CGI is that it doesn't provide for real persistence. Sure, you can use cookies, file-based data on the server side or append horribly long strings to the URL, but none of those is an ideal solution. In addition, if you're loading things like inventory data from a database, you have to account for initialization time and overhead every time the script is run.

In some situations, the best solution is to have a secondary server process running that shares data with Tcl through sockets in a true client-server fashion. In that way, your server could load the needed information and become a persistent data store, for whatever purpose. In Tcl, that's an easier task than you might expect. While the actual code is too lengthy to include in this article, I'll include an overview here and will be happy to provide additional details by e-mail (bills@multimedia.com).

Sockets in Tcl are designed to be easy to use. The socket command is used by applications wanting to establish a listening post on a port, as well as clients that want to connect to any server, Tcl or otherwise. How would you listen on a port? Just use:

socket -server sayHello 9999

Now you have a server listening on port 9999 that will execute the Tcl procedure sayHello whenever a new client connects. What if you want an asynchronous socket? Use:

socket -server -async sayHello 9999

socket 10.0.0.1 9999

So, once you've decided on what your server will do, your CGI program can parse the data as usual, quickly establish a socket connection, and then let the server process the information.