Rapid Prototyping with Tcl/Tk

A discussion of rapid prototyping and how it can benefit programmers in creating software to match the customer's needs.
An Example

The following example is meant to give you a feeling for what rapid prototyping is like. To keep it short and understandable it does not use any of the features of Tk at all. Although a graphical example would certainly be better suited to show the advantages of RP, the concepts presented can easily be extended to make use of Tk's extra features. It is one of my actual projects, and I will show it to you in the same way I developed it. As you may guess the customer and the programmer are in fact just one person—me.

Customer: “I need a nifty little library tool to work with values that are configurable by the end user. The values should come from a simple text file, and that's all there is to it.”

Programmer: “Okay. This one is really easy.”

I sit down and type some lines of Tcl code that give me the following interface:

proc Cfg <

This way I can use the configuration interface for a (hypothetical) text editor in this manner:

set Lines [Cfg "NumLines"]
The format of the file should be kept simple, so I try the following to read values from it:
NumLines 20
WordWrap 1
Reading the file into memory would need another function. As I don't want to pollute name spaces (true for both Tcl and C), I modify the signature of my procedure to be the following:
proc Cfg <
which gives me the following:
Cfg read "myconfig.cfg"
Cfg get NumLines
Code to implement all of this is written in 15 minutes. I add another 15 minutes to write some test cases so I can always verify the correctness of the whole implementation. Here is one test:
Test c1 "get simple value" {
  Cfg read "test.cfg"
  Cfg get NumLines
} 20
It reads my simple test configuration file and checks if the value for NumLines is really the same as set in the configuration file. It is all finished in 30 minutes. Sure, I have to rewrite it in C, but first I'll cross check with my customer.

Customer: “Nice work, but you see, if an application gets bigger, we may have name clashes—consider two modules that use a configuration value named Width. One could be the width of a window and the other the width of the text in the editor. What we really need is a module dependent solution.”

Programmer: “Grrr. I should have known—simple things tend to get complicated with time.”

Modifying the interface is straightforward, but what about the file format? Far back in my mind is a little hint named win.ini. I cross check with a real windows installation on my DOS partition—yes, the format would be just right for my problem, and it will already be familiar to a lot of users.

So, I modify my configuration file to look like this:

; my config test file: 24/02/1997
[Editor]
WordWrap=1
NumLines=20
Width=80
[Window]
Width=320
Height=400
; EOF

Reading such a file is far more complicated than the simple approach taken earlier. I pat myself on the shoulder that I have not yet written any C code.

Using some of the extra features of Tclx, an extension to Tcl, the file reader is ready and working in half an hour.

Now, I modify the interface as follows:

Cfg read "myconfig.cfg"
Cfg get Editor NumLines
Cfg get Editor Width
Cfg get Window Width

and I modify all the test cases. Again, I present the whole thing to my customer.

Customer: “Yeah, that's just what I wanted. I have some real world examples, could you verify that they work correctly?”

Programmer: The customer gives me some real X Window System configuration files—did you think the above were real world examples? Anyway, I run some tests on them and poof, all my nice looking code breaks. A quick look at the files reveals the problems—backslashes, brackets, blanks and entries that span multiple lines. Parsing an X configuration file is apparently more complicated than I thought at first.

I add a lot of new test cases that show all the above failures. They help me to make sure that I have a working piece of code at last, without one change breaking something that worked previously.

Adapting the Tcl code is comparatively easy. It takes some time though, as I have to fiddle around a bit with regular expressions for the parser. As soon as all the tests complete without error, I show my work to the customer.

Customer: “While waiting, I have used your code in a real world program, and it has one big drawback. Consider the following example configuration:

lpr -#1 -Plj5 myfile.txt

This is a print command which should be executed with exec. Some parts are static but depend on the operating system (e.g., lpr versus lp), others (such as the selection of the printer) may be done by the user and still others (such as the file name) come directly from the program. What I need is a flexible substitution scheme for configuration values.”

Programmer: “Ha. Now that's a really big request. How do I create such a flexible substitution scheme?”

It takes me a day of thinking (actually this was handled by a background process in my brain) to find the solution—Tk uses substitution for its key binding mechanism, so why not use the same ideas here?

Again, I modify the interface, so it can process code such as the following:

Cfg get Printer Command\
   "%c=1" "%f=myfile.txt"

and I add some lines to my configuration file:

[Printer]
Command= lpr "#%c "Plj5 %f
Before returning the final value to the caller my routine substitutes all “%?” sequences with the corresponding values from the argument list. Using Tcl's regular expressions makes this quite simple, and the only thing that takes some time is writing the tests to check for all these crazy conditions like “%%” and “\%”.

The above example will return

lpr -#1 -Plj5 myfile.txt

just as expected by the customer.

Customer: “Great. This is what I have been looking for for years. There's just one sub-optimal item left. If the user accidentally deletes his configuration file, the program will no longer work. Isn't it possible to keep some default values for this case?”

Programmer: “I have already thought of this situation. The code would be more readable if an actual value were present, and it could be tested more easily as I wouldn't have to write configuration files all the time.”

I modify the interface once more by adding a new subcommand:

Cfg def <module> <name> \
   <default> ?<sub>? ..."

The general syntax of the get and def sub-commands are the same (and both return the same configuration value when called with the same substitution values), but def also sets a default value when none is given.

If a configuration file has been read by Cfg, the values in this file take precedence, otherwise the default is used.

Here is an example:

Cfg def Printer Command \
   "lpr -#%c -Plj5 %f" \
   "%c=1" \
   "%f=myfile.txt"

This command returns:

lpr -#1 -Plj5 myfile.txt
Customer: “I use your code in all of my Tcl applications, and it works like a charm. The only thing I noticed is that the programs are really slow on startup. I suspect this is related to your configuration code. Couldn't you do anything about that?”

Programmer: “Aaargh.”

Back in the configuration business. First, I check some of the customers applications. No wonder he has slow startup—he uses configurations quite excessively, even for language internationalization. I instrument the applications for profiling and create some snapshots for analysis. The resulting diagrams (see Figure 1 and Figure 2) clearly state the two highest CPU-cycle-eaters: reading configuration files and substitution of configuration values.

Figure 1. Tcl Profiler Bar Chart

Figure 2. Tcl Profiler Graph

The interface is comparatively stable now (the customer has already used it in all his programs, he won't change it that easily). I start rewriting some of the code in C. As I know the exact bottlenecks from the analysis, the first two C routines replace the configuration reader and the substitution engine.

Now all of the previously written tests come in handy—I can check if the new code works correctly with nearly no effort. Both of the routines are quite a bit of work as they do some very tricky operations and pointers always point to somewhere unexpected. Sure it's nice to have a debugger for Tcl (see Figure 3), but without a C debugger, the life of a C programmer would be truly difficult.

Figure 3. Tcl Debugger

The new code is worth the price. Speed can be 5 to 10 times that of the original code, and this can mean the difference between a one second startup and a ten second startup.

There is a good deal more needed to make this little example into a full-fledged library, but it should be enough to see the general concepts. Here are the most important points to learn about the advantages of RP:

  • Work can begin with incomplete specifications.

  • Different implementations can be created in a straightforward and easy way using Tcl, until the customer is satisfied.

  • Working examples can be presented long before the application is complete.

  • If speed is a problem, the relevant parts can be rewritten in C.

  • Quality increases with added test procedures.

I did a really big project that integrated CAD (computer aided drafting) and PPC (production planning and control) software, and I successfully used the ideas presented above. (See Figure 4 and Figure 5.) The project took more than a year, and the specification changed more than once. The final solution shed another light on the versatility of Tcl: the user is able to define his own components (window frames, automatic actuators, etc.) using Tcl and its object-oriented extensions. This feature was just an offspring of our rapid prototyping philosophy as we simply provided the customer with our own tools.

Figure 4. PPC Software Screen

Figure 5. A Second PPC Software Screen

Using Tcl for software development is not the ultimate bells and whistles solution. There are drawbacks such as the lack of real data types (anything is a string), the inefficiency of interpreted code (although a compiler is now available), and the fact that eval stays a mystery even to Tcl fanatics. Tcl shines when your program is mainly centered on a graphical user interface. Other tasks may be done better by Perl, C or even FORTRAN. As dynamically loaded, shared libraries are now quite common on several platforms, we can split problems into chunks and solve these chunks with the language that is most appropriate for it. Tcl/Tk may be the glue for the graphical interface in this scenario.

What is Tcl/Tk?

What is Rapid Prototyping?

Richard Schwaninger (risc@finwds01.tu-graz.ac.at) (risc@ping.at) creates software for product automation systems. He uses Linux as his main development platform and builds tools for Tcl/Tk. He has been in the software business since the days of CP/M and has done a lot of late night hacking with AutoCad Lisp. He is married and lives in Graz, the capital of Styria/Austria, and he likes outdoor activities such as climbing, skiing and volleyball.

______________________

Comments

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

Re: Rapid Prototyping with Tcl/Tk

Anonymous's picture

I am balaji - Bangalore - INDIA. We use tcl/tk for our telecom projects. Your explanation and ilustration about the tcl/tk was impressive.
I need a favour. Could you please provide information about the software productivity for tcl/tk i.e line of code(LOC/per day) developed using tcl/tk per day. Is there any Industry standard benchmark for this.

Webinar
One Click, Universal Protection: Implementing Centralized Security Policies on Linux Systems

As Linux continues to play an ever increasing role in corporate data centers and institutions, ensuring the integrity and protection of these systems must be a priority. With 60% of the world's websites and an increasing share of organization's mission-critical workloads running on Linux, failing to stop malware and other advanced threats on Linux can increasingly impact an organization's reputation and bottom line.

Learn More

Sponsored by Bit9

Webinar
Linux Backup and Recovery Webinar

Most companies incorporate backup procedures for critical data, which can be restored quickly if a loss occurs. However, fewer companies are prepared for catastrophic system failures, in which they lose all data, the entire operating system, applications, settings, patches and more, reducing their system(s) to “bare metal.” After all, before data can be restored to a system, there must be a system to restore it to.

In this one hour webinar, learn how to enhance your existing backup strategies for better disaster recovery preparedness using Storix System Backup Administrator (SBAdmin), a highly flexible bare-metal recovery solution for UNIX and Linux systems.

Learn More

Sponsored by Storix