Indexing Texts with SMART
I do not use SMART for bread-and-butter retrieval, but for the weights it computes and the indices it creates. At this point I usually want to do some other manipulations of the data. I offer my thanks to the developers of Unix in general and to Linux in particular for creating a whole string of ever more complicated and sophisticated shell scripts, the standard Unix tools and a few of “My Very Own” utilities that suffice to process the SMART output to a file that is ready for import into SPSS.
Now I have to quit Linux and boot MS-DOS, start MS Windows and finally enter SPSS to do the statistics and create some graphs. I am a newcomer to Unix (indeed it was the fact that Linux offered a way to use SMART that pulled me over the line two years ago). While MS Windows is not my favorite operating system, SPSS gets the job done. When the output is written to disk, I immediately escape back to Linux to write the final article, report, or whatever with LaTeX.
On this point I have two messages—one bad. The good news is that SMART is obtainable by anonymous ftp from Cornell University and can be used free for scientific and experimental purposes. Better yet, it compiles under Linux without much tweaking and twiddling. There is also a fairly active mailing list for people who use SMART (firstname.lastname@example.org).
The bad news: the manual—what manual? SMART is not for the faint of heart; after unpacking and compilation, you'll find some extremely obscure notes and examples, and that is all. Nevertheless, if you have more than just a few megabytes of text to manage and the stamina to learn SMART, it certainly is the best solution for your information retrieval needs. I do wish someone would write a comprehensive manual. In the meantime, you may be helped by my “tutorial for newbies” found at http://pi0959.kub.nl:2080/Paai/Onderw/Smart/hands.html.
This article was published previously in Issue 13 of the Linux Gazette.
Hans “Paai” Paijmans (email@example.com) is a University lecturer and researcher at Tilburg University and a regular contributor to several Dutch journals. Together with E. Maryniak, he wrote the first Dutch book on Linux—already two years ago. My, doesn't the time fly? His home page is at http://pi0959.kub.nl:2080/paai.html .
- Readers' Choice Awards 2013
- IBM Will Minimize Impact of Future Disasters
- RSS Feeds
- New Products
- Sublime Text: One Editor to Rule Them All?
- December 2013 Issue of Linux Journal: Readers' Choice
- Linux Systems Administrator
- Senior Perl Developer
- Technical Support Rep
- Raspberry Pi: the Perfect Home Server
- Personally, I am no longer a
37 min 44 sec ago
- Reply to comment | Linux Journal
14 hours 2 min ago
- It's Jupiter
15 hours 2 min ago
- GIMP is certainly a graphic
16 hours 4 min ago
- Thanks For Your Sharing
20 hours 50 min ago
- Studying linux, and looking
1 day 15 min ago
- voting for Best Linux Distribution
1 day 8 hours ago
- tizen vs android
1 day 12 hours ago
- i switch my choice from KDE
1 day 13 hours ago
1 day 16 hours ago