Indexing Texts with SMART
I do not use SMART for bread-and-butter retrieval, but for the weights it computes and the indices it creates. At this point I usually want to do some other manipulations of the data. I offer my thanks to the developers of Unix in general and to Linux in particular for creating a whole string of ever more complicated and sophisticated shell scripts, the standard Unix tools and a few of “My Very Own” utilities that suffice to process the SMART output to a file that is ready for import into SPSS.
Now I have to quit Linux and boot MS-DOS, start MS Windows and finally enter SPSS to do the statistics and create some graphs. I am a newcomer to Unix (indeed it was the fact that Linux offered a way to use SMART that pulled me over the line two years ago). While MS Windows is not my favorite operating system, SPSS gets the job done. When the output is written to disk, I immediately escape back to Linux to write the final article, report, or whatever with LaTeX.
On this point I have two messages—one bad. The good news is that SMART is obtainable by anonymous ftp from Cornell University and can be used free for scientific and experimental purposes. Better yet, it compiles under Linux without much tweaking and twiddling. There is also a fairly active mailing list for people who use SMART (firstname.lastname@example.org).
The bad news: the manual—what manual? SMART is not for the faint of heart; after unpacking and compilation, you'll find some extremely obscure notes and examples, and that is all. Nevertheless, if you have more than just a few megabytes of text to manage and the stamina to learn SMART, it certainly is the best solution for your information retrieval needs. I do wish someone would write a comprehensive manual. In the meantime, you may be helped by my “tutorial for newbies” found at http://pi0959.kub.nl:2080/Paai/Onderw/Smart/hands.html.
This article was published previously in Issue 13 of the Linux Gazette.
Hans “Paai” Paijmans (email@example.com) is a University lecturer and researcher at Tilburg University and a regular contributor to several Dutch journals. Together with E. Maryniak, he wrote the first Dutch book on Linux—already two years ago. My, doesn't the time fly? His home page is at http://pi0959.kub.nl:2080/paai.html .
|Reglue: Opening Up the World to Deserving Kids, One Linux Computer at a Time||Jul 29, 2014|
|diff -u: What's New in Kernel Development||Jul 23, 2014|
|Great Scott! It's Version 13!||Jul 21, 2014|
|Adminer—Better Than Awesome!||Jul 17, 2014|
|It Actually Is Rocket Science||Jul 16, 2014|
|Android Candy: Repix, Not Just Another Photo App||Jul 14, 2014|
- Reglue: Opening Up the World to Deserving Kids, One Linux Computer at a Time
- Numerical Python
- Download "Linux Management with Red Hat Satellite: Measuring Business Impact and ROI"
- NSA: Linux Journal is an "extremist forum" and its readers get flagged for extra surveillance
- diff -u: What's New in Kernel Development
- Readers' Choice Awards 2013
- RSS Feeds
- Linux Systems Administrator
- Senior Perl Developer
- Technical Support Rep