Work the Shell - Pushing Your Message Out to Twitter

November 1st, 2008 by Dave Taylor in

Use the shell to generate movie trivia from a movie database.
Your rating: None Average: 4 (4 votes)

Holy cow, have we really been working on the movie trivia Twitter stream for almost a year now? This surely must be the longest time-per-line-of-code project in the history of software development.

Previous columns have combined to give us a set of shell scripts that scrape the Internet Movie Database (IMDb, www.imbd.com) for its top 250 list, then pull out the year of release for each film (inconveniently, not available on the same page as the top 250 list), randomly select reasonable incorrect guesses for release year and output the question in the form: “IMDB Top 250 Movie #81: Was 2001: A Space Odyssey released in 1968, 1973 or 1975?”

That's rather more work behind the scenes than you might realize. Of course, if you've been reading all the previous columns, you actually have firsthand knowledge of just how many hoops we've had to jump through (but fortunately, no “hogsheads of real fire”).

This month, we're going to turn our attention to the Twitter microblogging service, as we finally have the ability to produce the desired message. Now, we just need a mechanism to publish it to Twitter.

Hacking the Twitter API

I could take the long way and actually read through the Twitter API to learn how to make specific calls and interact with the service elegantly and appropriately, but that sounds like work, doesn't it?

It turns out, there's a simple way to invoke Twitter through a command-line Web utility and get access to much of the basic data. My tool of choice? Curl, a slick utility that makes it easier to work with Internet services through the command line.

The URL is twitter.com/statuses/update.json, but the way you pass data is a bit tricky. You need to send it as a name=value pair in the connection stream, not as a GET value or other URL-based technique. The variable it seeks is status, and you can test it by doing this:

$ curl --data-ascii status=testing \
http://twitter.com/statuses/update.json

The problem is immediately obvious in the return message:

Could not authenticate you.

Ah, well, yes, we haven't specified our user ID. Those credentials aren't sent via the URL or --data-ascii, but instead through a basic HTTP auth. So, in fact, you need to do this:

$ curl --user "$user:$pass" --data-ascii \
status=testing "http://twitter.com/statuses/update.json" 

Now, of course, you need a Twitter account to utilize. For this project, I'm going to continue working with FilmBuzz, a Twitter account that I also set up to disseminate interesting film-industry news.

For the purposes of this article, let's assume the password is DeMille, though, of course, that's not really what it is. The user and pass variables can be set, and then we invoke Curl to see what happens:

$ curl --user "$user:$pass" --data-ascii status=testing
http://twitter.com/statuses/update.json
{"truncated":false,"in_reply_to_status_id":null,"text":
↪"testing","favorited":null,"in_reply_to_user_id":
↪null,"source":"web","id":880576363,"user":{"name":
↪"FilmBuzz","followers_count":214,"url":null,
↪"profile_image_url":"http:\/\/s3.amazonaws.com\
↪/twitter_production\/profile_images\/55368346\
↪/FilmReelCloseUp_normal.JPG","description":"Film
↪trivia game, coming soon!","location":"Hollywood, of
↪course","screen_name":"FilmBuzz","id":15097176,"protected":
↪false},"created_at":"Thu Aug 07 16:51:49 +0000 2008"}

The return data gives you a nice sneak peek into the configuration of the FilmBuzz twitter account and tells you that what we sent wasn't too long (truncated:false), but otherwise, it's pretty forgettable stuff, isn't it?

How do you ignore output with a command in the shell? Reroute the output to /dev/null, of course:

$ curl --user "$user:$pass" --data-ascii status=testing
http://twitter.com/ses/update.json > /dev/null
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 505 100 505 0 0 1127 0 --:--:-- --:--:-- --:--:-- 0

Nope, that doesn't work, because then Curl wants to give us some transactional stats. How do we mask that? Use the --silent flag to Curl. Now, we're just about there:

$ curl --silent --user "$user:$pass" --data-ascii status=testing
http://twitter.com/statuses/update.json > /dev/null
$

That's it. Let's put that into a shell script, so we can simply invoke tweet with the message we want to send out to followers of FilmBuzz. (And, if you aren't yet following FilmBuzz on Twitter, why the heck not? Go to twitter.com/FilmBuzz, and click follow.) Actually, we need to take into account one more thing: although our messages on the command line are quite likely to have spaces within them, we can't send a name=value pair with spaces. Instead, a quick invocation of tr lets us convert all spaces into + signs, the commonly accepted HTTP encoding:

#!/bin/sh
# Twitter command line interface
user="FilmBuzz" ; pass="DeMille"
curl="/usr/local/bin/curl"

$curl --silent --user "$user:$pass" --data-ascii \
"status=$(echo $@ | tr ' ' '+')" "http://twitter.com/
↪statuses/update.json"
> /dev/null
exit 0

This isn't a particularly robust solution, because what happens if we have a + character in the message we want to transmit? It gets lost. That can be addressed by first checking to see whether there are + signs and converting them to the safe HTTP-encoding equivalent, %2B. You can't do that with tr, however, because it wants 1:1 substitution, so we'll use sed instead and pull the substitution onto its own line for better style too:

msg=$(echo $@ | sed 's/+/%2B/g;s/ /+/g')

$curl --silent --user "$user:$pass" --data-ascii \
"status=$msg" "http://twitter.com/statuses/update.json" > /dev/null

Problem solved. Now you can do simple things like:

$ tweet 'Tweeting from the command line? Well, sure!'

Nice. Now, to put everything together. Oh. We've run out of space. Again. Okay, next month. We finally have all the building blocks we need. Remember, sign up for Twitter, and then follow @FilmBuzz, so you can see the fruit of this work.

Dave Taylor is a 26-year veteran of UNIX, creator of The Elm Mail System, and most recently author of both the best-selling Wicked Cool Shell Scripts and Teach Yourself Unix in 24 Hours, among his 16 technical books. His main Web site is at www.intuitive.com, and he also offers up tech support at AskDaveTaylor.com. Follow Dave on Twitter through twitter.com/DaveTaylor.

__________________________


Special Magazine Offer -- Free Gift with Subscription
Receive a free digital copy of Linux Journal's System Administration Special Edition as well as instant online access to current and past issues. CLICK HERE for offer

Linux Journal: delivering readers the advice and inspiration they need to get the most out of their Linux systems since 1994.

Post new comment

Please note that comments may not appear immediately, so there is no need to repost your comment.
The content of this field is kept private and will not be shown publicly.
  • Allowed HTML tags: <a> <em> <strong> <cite> <code> <pre> <ul> <ol> <li> <dl> <dt> <dd> <i> <b>
  • Lines and paragraphs break automatically.

More information about formatting options

Newsletter

Each week Linux Journal editors will tell you what's hot in the world of Linux. You will receive late breaking news, technical tips and tricks, and links to in-depth stories featured on www.linuxjournal.com.
Sign up for our Email Newsletter

Tech Tip Videos

From the Magazine

December 2009, #188

If last month's Infrastrucuture issue was too "big" for you then try on this month's Embedded issue. Find out how to use Player for programming mobile robots, build a humidity controller for your root cellar, find out how to reduce the boot time of your embedded system, and if you're new to embedded systems find out the basics that go into one. You can also read about the Beagle Board, the Mesh Potato and a spate of other interestingly named items. And along with our regular columns don't miss our new monthly column: Economy Size Geek.







Read this issue