Work the Shell - Exploring Lat/Lon with Shell Scripts

 in
Never get lost at the command line again.

With the rise of geolocation systems on mobile devices (think “around me” on the Apple iPhone), a consistent method of measuring points on Earth has become quite important. The standard that's used is latitude and longitude, which measure the distance north or south of the equator and the distance east or west of the prime meridian (which goes through Greenwich, England). Your GPS devices all understand this notation, as does Google Maps, Yahoo Maps, MapQuest and so on.

From a shell scripting perspective, we're interested in both being able to identify lat/lon for a point on the Earth and then, armed with that information, to see if we can calculate the distance between two points on the planet.

The first seems almost insurmountably hard until you learn that Yahoo Maps has a very simple API that lets you specify a URL that includes a street address and returns an XML object that includes its lat/lon values.

Where Is This Place?

For example, you might be familiar with 1600 Pennsylvania Avenue, Washington, DC. I know you've seen pictures of the place. What's its lat/lon?


$ u='http://api.maps.yahoo.com/ajax/geocode'
$ a='?appid=onestep&qt=1&id=m&qs=1600+pennsylvania+ave+washington+dc'
$ curl "$u$a"
YGeoCode.getMap({"GeoID"      : "m",
                 "GeoAddress" : "1600 pennsylvania ave washington dc",
                 "GeoPoint"   : {"Lat" : 38.89859,
                                 "Lon" : -77.035971},
                 "GeoMID"     : false,
                 "success"    : 1} ,1);
<!-- xm6.maps.re3.yahoo.com uncompressed/chunked
     Tue Aug  4 12:16:51 PDT 2009 -->

Note that the output actually comes back as two lines; the the data above, and in the other examples, has been reformatted to make it more readable.

Skim that return object, and you'll see Latitude = 38.89859 and Longitude = -77.035971. Feed those two into Google Maps as “38.89859,-77.035971” as a check, and you'll find the image shown in Figure 1.

Figure 1. The White House

You guessed it, it's the street address of the White House.

Let's start by creating a simple script where you can specify a street address and it will output lat/lon values.

Scripting Our Solution

The first part is easy: take whatever was specified on the command line, and “recode” it to be URL-friendly. Then, append that to the Yahoo API URL, and output the results of a curl call:


#!/bin/sh

url='http://api.maps.yahoo.com/ajax/geocode'
args='?appid=onestep&qt=1&id=m&qs='
converter="$url$args"

addr="$(echo $* | sed 's/ /+/g')"
curl -s "$converter$addr"
exit 0

Let's test it with a different address this time:


$ sh whereis.sh 2001 Blake Street, Denver, CO
YGeoCode.getMap({"GeoID"      : "m",
                 "GeoAddress" : "2001 Blake Street, Denver, CO",
                 "GeoPoint"   : {"Lat" : 39.754386,
                                 "Lon" : -104.994261},
                 "GeoMID"     : false,
                 "success"    : 1}, 1);
<!-- x1.maps.sp1.yahoo.com uncompressed/chunked
     Tue Aug  4 12:37:44 PDT 2009 -->

You can figure out what's at this address if you like. More important, you can see that this simple four-line script does the job—sort of.

Cleaning Up the Output

What we really want, however, is to extract just the lat and lon values and toss everything else out. This can be done with a bunch of different tools, of course, including Perl and awk, but I'm a rebel, so I use cut instead.

To do this, we need to count the double quotes (") in the output block. The 12th double quote is immediately before the latitude value, and the 15th is immediately after the longitude value. If we just worked with that, we would get:

$ sh whereis.sh 2001 Blake Street, Denver, CO | cut -d\" -f13-15
:39.754386,"Lon":-104.994261},

Okay, so that's most of the work. Better, though, is to specify two different specific fields (13,15 rather than 13-15):

$ sh whereis.sh 2001 Blake Street, Denver, CO | cut -d\" -f13,15
:39.754386,":-104.994261},

That's 99% of what we want. Now we just need to clean up the noise. To do that, I'll jump back into the script itself, rather than experimenting on the command line:

curl -s "$converter$addr" | \
    cut -d\" -f13,15 | \
    sed 's/[^0-9\.\,\-]//g'

And testing:

$ sh whereis.sh 2001 Blake Street, Denver, CO
39.754386,104.994261,

Almost. Really, really close. But, that last comma is not wanted. Hmmm....

Okay! To delete the last comma, we simply need to add a second substitution to the sed statement, so that the full sed expression is now:

sed 's/[^0-9\.\,\-]//g;s/,$//'

______________________

Dave Taylor has been hacking shell scripts for over thirty years. Really. He's the author of the popular "Wicked Cool Shell Scripts" and can be found on Twitter as @DaveTaylor and more generally at www.DaveTaylorOnline.com.

Comments

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

error?

Anonymous's picture

When I run this script I get:
whereis: 7: Syntax error: Unterminated quoted string

Here is the script as I have it:

#!/bin/sh

url='http://api.maps.yahoo.com/ajax/geocode'
args='?appid=onestep&qt=1&id=m&qs="
converter="$url$args"

addr="$(echo $* | sed 's/ /+/g')"
  curl -s "$CONVERTER$ADDR" | \
      sed -e 's/.*{\("Lat":[^}]*\).*/\1/' -e 's/"L.."://g'
exit 0

Here is the command I'm running:
sh whereis 2001 Blake Street, Denver, CO

This seems pretty simple but I cannot see what I'm missing.

Cupla Problems

Mitch Frazier's picture

You've got a couple problems in your script, the one that's giving you the error is here:

args='?appid=onestep&qt=1&id=m&qs="

You've got a string with mismatched quotes: it starts with a single quote and ends with a double quote, you need this:

args='?appid=onestep&qt=1&id=m&qs='
#                                 ^

or this

args="?appid=onestep&qt=1&id=m&qs="
#    ^

The other problem is that the shell is case sensitive, so you can't use "converter" and "CONVERTER" interchangeably, pick one. Same with "addr" and "ADDR". So change:

  curl -s "$CONVERTER$ADDR" | \

to

  curl -s "$converter$addr" | \

p.s. The first error (the mismatched quotes) was in the original text, it has now been fixed.

Mitch Frazier is an Associate Editor for Linux Journal.

What about bad addresses?

Grishnakh's picture

Is there any way to tell if the address you input is invalid? I've tried a few made-up bad addresses and it still seems to give valid lat/lon coordinates.

bad address

Anonymous's picture

That's a "Feature" of yahoo's service. They'll give you their best estimate. If the house number is bad, they'll tell you the location of the street; if the street is bad, they'll tell you the location of the city; if the city is bad, they'll tell you the location of the state....

To tell if the address is bad you could, perhaps, compare the results with the results of a less specific look up.

shell errors

zeugma's picture

As written, I get only longitude.

when I change the sed to...

    cut -d\" -f11,13 | \

I get the expected output

$ cat whereis.sh
#!/bin/bash

URL='http://api.maps.yahoo.com/ajax/geocode'
ARGS='?appid=onestep&qt=1&d=m&qs='
CONVERTER="$URL$ARGS"
ADDR="$(echo $* | sed 's/ /+/g')"

curl -s "$CONVERTER$ADDR" | \
    cut -d\" -f11,13 | \
    sed 's/[^0-9\.\,\-]//g;s/,$//'
exit 0
$ ./whereis.sh 1313 S Disneyland Drive, Anaheim, CA
33.814413,-117.924424

Response Is Different

Mitch Frazier's picture

The response you're getting is different, the GeoID value is not a quoted value as above. The following would avoid problems related to quotes:

  curl -s "$CONVERTER$ADDR" | \
      sed -e 's/.*{\("Lat":[^}]*\).*/\1/' -e 's/"L.."://g'

Mitch Frazier is an Associate Editor for Linux Journal.

Webinar
One Click, Universal Protection: Implementing Centralized Security Policies on Linux Systems

As Linux continues to play an ever increasing role in corporate data centers and institutions, ensuring the integrity and protection of these systems must be a priority. With 60% of the world's websites and an increasing share of organization's mission-critical workloads running on Linux, failing to stop malware and other advanced threats on Linux can increasingly impact an organization's reputation and bottom line.

Learn More

Sponsored by Bit9

Webinar
Linux Backup and Recovery Webinar

Most companies incorporate backup procedures for critical data, which can be restored quickly if a loss occurs. However, fewer companies are prepared for catastrophic system failures, in which they lose all data, the entire operating system, applications, settings, patches and more, reducing their system(s) to “bare metal.” After all, before data can be restored to a system, there must be a system to restore it to.

In this one hour webinar, learn how to enhance your existing backup strategies for better disaster recovery preparedness using Storix System Backup Administrator (SBAdmin), a highly flexible bare-metal recovery solution for UNIX and Linux systems.

Learn More

Sponsored by Storix