Shell Scripting with a Distributed Twist: Using the Sleep Scripting Language
which is equal to this Java statement:
System.out.println("Hello World");
When calling into Java, the message is the name of a method or field that belongs to the object. Arguments are converted to Java types as necessary, and some conversions are automatic. Nearly anything will convert to a string. However, a string will not convert to an int. Casting is possible, but I don't cover that topic here.
Now that you know a little about the Sleep language, it helps to see it in action. Next, I present several scenarios and Sleep-based solutions to them.
My home directory has many files. I'm a digital packrat, and I'm always low on disk space. I really have no idea what is on my disk. To help, I wrote a script to find the largest files within a directory and its subdirectories:
global('$size $file @files %sizes');
sub processFile
{
This script creates a data structure of files and their sizes, sorts it, and presents the results to the user. The &processFile function does most of the work, and it expects a file as an argument:
if (-isDir $1)
{
filter(&processFile, ls($1));
}
If the argument is a directory, the &ls function will provide the contents of the directory as an array. &filter expects a function and an array as arguments. &filter calls the function on each item in the array. I use &filter to call &processFile on the argument's subdirectories and files:
else if (lof($1) > (1024 * 1024))
{
%sizes[$1] = lof($1);
}
}
The hash %sizes stores each filename and size. The key is the filename, and the size is the value. The &lof function returns the length of a file in bytes. I ignore files smaller than 1MB in size. I have so many files that this script exhausts the memory of Java before finishing. I could set Java to use a larger heap size with java -Xmx1024M -jar sleep.jar. Below, I chose to fix my script:
processFile(@ARGV[0]);
I call &processFile on the first command-line argument to kick off the script. When this function returns, the %sizes hash will contain an entry for each file in the specified directory and its subdirectories:
@files = sort({ return %sizes[$2] <=> %sizes[$1]; },
keys(%sizes));
The &sort function processes the keys of %sizes and places them in order from largest to smallest size. Much like Perl, Sleep's &sort can use any criteria given by an anonymous function:
foreach $file (sublist(@files, 0, 50))
{
$size = lof($file);
println("$[20]size $file");
}
This script ends with a foreach loop to print out the 50 largest files.
And, lo and behold! I solved my problem. I found four copies of a Christmas movie I made on my Macintosh three years ago. Thanks to the script, I recovered several gigabytes of disk space.
Recently, I had to watch this movie about a guy who sent letters to his wife after he passed away. I'm not really into the romantic-morbid genre; however, I thought I could show the people in my life how much I care about them. Here is a script that sends a random fortune to someone every 24 hours:
include("sendemail.sl");
while (1)
{
sendemail($to => "rsmudge@gmail.com",
$from => "raffi@hick.org",
$subject => "P.S. I love you",
$message => "This made me think of you:\n\n" .
join("\n", `fortune`)
);
# sleep for 24 hours
sleep(24 * 60 * 60 * 1000);
}
I use `fortune` to execute the fortune command and collect its output into an array. Then, I combine this with the rest of the message body to make a thoughtful message. This script uses the $variable => value syntax to pass named arguments to &sendemail.
Backticks are one way to execute a process. I show the other way in the sendemail.sl code.
I use the sendmail program to send e-mail. The sendemail.sl file contents are:
sub sendemail
{
local('$handle');
$handle = exec("/usr/sbin/sendmail -t $to");
Sleep executes processes with the &exec function. Scripts interact with processes as if they were files. As an aside, you can pass arguments with spaces to &exec. Use an array instead of a string. For example, exec(@("/usr/sbin/sendmail", "-t", $to)) would work in this example:
println($handle, "TO: $to FROM: $from SUBJECT: $subject $message");
Here, I send the e-mail message to the sendmail process over STDIN. Later in this article, I cover how to use Sleep for distributed tasks. Don't combine this e-mail example with that—I don't like spammers:
closef($handle); }
The last step is to close the handle. Having successfully automated my personal life, let's turn our attention to work matters.
Realizing the promise of Apache® Hadoop® requires the effective deployment of compute, memory, storage and networking to achieve optimal results. With its flexibility and multitude of options, it is easy to over or under provision the server infrastructure, resulting in poor performance and high TCO. Join us for an in depth, technical discussion with industry experts from leading Hadoop and server companies who will provide insights into the key considerations for designing and deploying an optimal Hadoop cluster.
Sponsored by AMD
Built-in forensics, incident response, and security with Red Hat Enterprise Linux 6
Every security policy provides guidance and requirements for ensuring adequate protection of information and data, as well as high-level technical and administrative security requirements for a system in a given environment. Traditionally, providing security for a system focuses on the confidentiality of the information on it. However, protecting the data integrity and system and data availability is just as important. For example, when processing United States intelligence information, there are three attributes that require protection: confidentiality, integrity, and availability.
Learn more about catching the bad guy in this free white paper.
Sponsored by DLT Solutions
| Designing Electronics with Linux | May 22, 2013 |
| Dynamic DNS—an Object Lesson in Problem Solving | May 21, 2013 |
| Using Salt Stack and Vagrant for Drupal Development | May 20, 2013 |
| Making Linux and Android Get Along (It's Not as Hard as It Sounds) | May 16, 2013 |
| Drupal Is a Framework: Why Everyone Needs to Understand This | May 15, 2013 |
| Home, My Backup Data Center | May 13, 2013 |
- New Products
- Linux Systems Administrator
- Senior Perl Developer
- Technical Support Rep
- Web & UI Developer (JavaScript & j Query)
- UX Designer
- Designing Electronics with Linux
- Dynamic DNS—an Object Lesson in Problem Solving
- Using Salt Stack and Vagrant for Drupal Development
- Making Linux and Android Get Along (It's Not as Hard as It Sounds)
Enter to Win an Adafruit Pi Cobbler Breakout Kit for Raspberry Pi

It's Raspberry Pi month at Linux Journal. Each week in May, Adafruit will be giving away a Pi-related prize to a lucky, randomly drawn LJ reader. Winners will be announced weekly.
Fill out the fields below to enter to win this week's prize-- a Pi Cobbler Breakout Kit for Raspberry Pi.
Congratulations to our winners so far:
- 5-8-13, Pi Starter Pack: Jack Davis
- 5-15-13, Pi Model B 512MB RAM: Patrick Dunn
- 5-21-13, Prototyping Pi Plate Kit: Philip Kirby
- Next winner announced on 5-27-13!
Featured Jobs
| Linux Systems Administrator | Houston and Austin, Texas | Host Gator |
| Senior Perl Developer | Austin, Texas | Host Gator |
| Technical Support Rep | Houston and Austin, Texas | Host Gator |
| UX Designer | Austin, Texas | Host Gator |
| Web & UI Developer (JavaScript & j Query) | Austin, Texas | Host Gator |
Free Webinar: Hadoop
How to Build an Optimal Hadoop Cluster to Store and Maintain Unlimited Amounts of Data Using Microservers
Realizing the promise of Apache® Hadoop® requires the effective deployment of compute, memory, storage and networking to achieve optimal results. With its flexibility and multitude of options, it is easy to over or under provision the server infrastructure, resulting in poor performance and high TCO. Join us for an in depth, technical discussion with industry experts from leading Hadoop and server companies who will provide insights into the key considerations for designing and deploying an optimal Hadoop cluster.
Some of key questions to be discussed are:
- What is the “typical” Hadoop cluster and what should be installed on the different machine types?
- Why should you consider the typical workload patterns when making your hardware decisions?
- Are all microservers created equal for Hadoop deployments?
- How do I plan for expansion if I require more compute, memory, storage or networking?




2 hours 55 min ago
8 hours 41 min ago
8 hours 58 min ago
10 hours 51 min ago
12 hours 45 min ago
19 hours 39 min ago
19 hours 55 min ago
21 hours 46 min ago
1 day 3 hours ago
1 day 8 hours ago