Home ›

Work the Shell - More Fun with Word and Letter Counts

Issue 180

From Issue #180
April 2009

Apr 01, 2009  By Dave Taylor
 in
Determine the frequency of letters within a document (and become unbeatable at Hangman).
______________________

Dave Taylor has been hacking shell scripts for over thirty years. Really. He's the author of the popular "Wicked Cool Shell Scripts" and can be found on Twitter as @DaveTaylor and more generally at www.DaveTaylorOnline.com.

Comments

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

Letter frequency

opie's picture
Submitted by opie on Fri, 03/20/2009 - 14:41.

Samuel Morse would have loved your article, as I did. In devising Morse code, he assigned the briefest codes to the most frequently used letters. Lacking your routine and access to /usr/share/dict/words, he based his count on the number of letters in sets of printers' type, according to AskOxford.com (http://www.askoxford.com/asktheexperts/faq/aboutwords/frequency). The figures he came up with for the most common letters were:

12,000 E
9,000 T
8,000 A, I, N, O, S

In contrast, Zs occupied the least letter space in typesetters' cabinets, with only 200 on hand.

As a result, E and T in Morse code require single key presses, one short and the other long (dot and dash, respectively). Other letters may require four key presses, combining dots and/or dashes. Optimization makes a difference; even so sending 5 words per minute in Morse code is a challenge for the novice; 20 words per minute the mark of a pro!

AskOxford.com poses some interesting questions that shell scripters could have fun with. For example,

Are there any English words containing the same letter three times in a row?

Are there any words in the English language that use all five vowels with no intervening consonants or have the five vowels in the right order?

Too easy? Try: What is the longest one-syllable English word?

I remember before you could look virtually everything up on the Internet, being surprised how difficult it was to code a routine for breaking words into syllables. Not because the coding itself was hard but because detailing the underlying rules was such a challenge.

Webcast
More Resources
How to Build an Optimal Hadoop Cluster to Store and Maintain Unlimited Amounts of Data Using Microservers

Realizing the promise of Apache® Hadoop® requires the effective deployment of compute, memory, storage and networking to achieve optimal results. With its flexibility and multitude of options, it is easy to over or under provision the server infrastructure, resulting in poor performance and high TCO. Join us for an in depth, technical discussion with industry experts from leading Hadoop and server companies who will provide insights into the key considerations for designing and deploying an optimal Hadoop cluster.

Learn More

Sponsored by AMD

White Paper
More Resources
Red Hat White Paper: Using an Open Source Framework to Catch the Bad Guy

Built-in forensics, incident response, and security with Red Hat Enterprise Linux 6

Every security policy provides guidance and requirements for ensuring adequate protection of information and data, as well as high-level technical and administrative security requirements for a system in a given environment. Traditionally, providing security for a system focuses on the confidentiality of the information on it. However, protecting the data integrity and system and data availability is just as important. For example, when processing United States intelligence information, there are three attributes that require protection: confidentiality, integrity, and availability.

Learn more about catching the bad guy in this free white paper.

Learn More

Sponsored by DLT Solutions