At the Forge - Google Web Services

With a little SOAP, cleanliness is next to Googliness.
SOAP::Lite

SOAP, formerly the Simple Object Access Protocol, but now an acronym that officially doesn't stand for anything, provides a relatively easy method for sending an XML-encapsulated query to a server. The server then responds with an XML-encoded response. Over the years, SOAP has strayed far from its simple roots. Although SOAP is still easier to understand, implement and work with than some more complicated protocols (such as CORBA), it is more difficult than most people would like to admit. If I can get away with it, I personally prefer to use XML-RPC for Web services. Although XML-RPC doesn't offer all of the features of SOAP, it is far easier to work with.

That said, Google requires that we use SOAP, and with many good SOAP client libraries available nowadays, we should not be afraid to work with it. Perl programmers have a particularly strong implementation, known as SOAP::Lite, at their disposal. For the programming examples in this article, we use Perl and SOAP::Lite. Note that the Lite part of the module name describes the ease with which programmers can implement Web services, not a stripped-down version of SOAP. You can install the latest version of SOAP::Lite from CPAN by typing:


perl -MCPAN -e 'install SOAP::Lite'

The SOAP::Lite installation will ask you to indicate which tests, if any, you want to perform before installing the module. I normally accept the defaults, but you might want to add to or remove from these depending on your needs.

With SOAP::Lite installed, it's time to write a program that queries Google. But to do that, we need to know the URL of the service, as well as the method that we will be invoking on Google's computer, along with the names and types of any parameters we want to send. We could specify these by hand, but that would mean a lot of work on our part. Moreover, Google currently expects SOAP requests to be pointed at api.google.com/search/beta2. If Google ever decides to change that URL without warning, many people might be surprised and upset.

Luckily, Google has provided a WSDL file, describing the services offered via Google's APIs, as well as the request and response parameters the system accepts. It also describes the endpoint for queries, allowing Google (in theory) to make changes to the service without notifying developers in advance. Of course, this assumes that the WSDL file itself will remain in the same location. It also assumes that the names of the services will not change, and that each of them is documented somewhere, because the choice of which method to invoke still requires human intervention.

WSDL is written in XML, and it is fairly easy to understand, once you realize that it's describing nothing more than the various Web services available on a particular server, including the number, names and types of inputs. Thus, the WSDL entry for doGoogleSearch, which performs the basic Google search of Web content, is defined as follows:


<message name="doGoogleSearch">
<part name="key"            type="xsd:string"/>
<part name="q"              type="xsd:string"/>
<part name="start"          type="xsd:int"/>
<part name="maxResults"     type="xsd:int"/>
<part name="filter"         type="xsd:boolean"/>
<part name="restrict"       type="xsd:string"/>
<part name="safeSearch"     type="xsd:boolean"/>
<part name="lr"             type="xsd:string"/>
<part name="ie"             type="xsd:string"/>
<part name="oe"             type="xsd:string"/>
</message>

To use WSDL from within a Perl program using SOAP::Lite, we invoke SOAP::Lite->service with the WSDL file's URL. If the file resides on the local filesystem, make sure that the URL begins with file:. For example:


my $google_wsdl = "http://api.google.com/GoogleSearch.wsdl";
my $query = SOAP::Lite->service($google_wsdl);

SOAP::Lite is then smart enough to look through the WSDL and make all of the advertised methods dynamically available, such that we can do the following:


my $results =
$query->doGoogleSearch($google_key,
               $query_string,
               $starting_page,
               $max_results,
               $filter,
               $geographic_restriction,
               $safe_search,
               $language_restriction, 'utf-8', 'utf-8');

Do you see what happened here? There is a one-to-one mapping between the inputs described in the WSDL and the parameters that we pass to $query->doGoogleSearch().

Simple Queries with doGoogleSearch

We have now seen the core of our Google search program written in Perl. All that's left is to review the input parameters and the contents of $results, which contains the results returned from Google.

The documentation for the API at www.google.com/apis/reference.html describes the input parameters. All of them are mandatory, but some of them are more important than others. In particular, the Google key and the query string typically will be set, and the others will be set with simple default values, as you can see in Listing 1.

______________________

Comments

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

Google won't give me the key and code for web page : Google maps

Derry Adams's picture

Google maps are terribly inaccurate.

Google DO NOT provide :
Advanced Geocoding
Option for Large Static Maps

service Level Agreement
Technical Support
Support portal & usage reporting

Google maps for business is a basic mapping service for API and is not always free. The customisation is poor, and the coverage is not great, seldom recognising small towns and only to postcode accuracy as NO geocoding !

If something is good enough, they would not give it away !

i prefer a Viamichelin solution, much more professional

Nick Wood's picture

Just had this back from a business who has moved to Google to ViaMichelin !

Initially Simplytrax used a product I trialed and it stank (2008) and I told them I can only assume that this is a joke.

They have sinced changed provider and told me that Google maps are terrible unlike our new system with benefits from enhanced ViaMichelin technology.
Google maps web service is about the worst mapping I have seen and I have seen most mapping systems. No reputable company would use Google API.

Our new platform uses a viamichelin business web service which far surpasses anything we have seen before.

More coverage, better accuracy and support and a world recognised brand for quality for a price less than I thought ( ViaMichelin were Cheaper than the Google enterprise Ajax API we solution by 25% with straight forward Javascript coding !

Mark Boardman of ViaMichelin business UK stated " We are pleased to have secured this contract to ensure the continued success of SimplyTrax"

Nick

Simply Trax

Google won't give me the key and code for web page

arlequin's picture

Dear Reuven,
Congratulations for your article!

I've registered a Google account this morning but still haven't received an email with the file... Is it normal this way?

I'd like to use your perl script embedded into a web page but I work with PHP. Is there a simple way to put that perl code to work on a web page assuming the parameters came from a form?
I assume PHP 4 manages SOAP, can I work with PHP/SOAP?

Thanks

Gabriel Menini
Linux Registered User #207262

White Paper
Linux Management with Red Hat Satellite: Measuring Business Impact and ROI

Linux has become a key foundation for supporting today's rapidly growing IT environments. Linux is being used to deploy business applications and databases, trading on its reputation as a low-cost operating environment. For many IT organizations, Linux is a mainstay for deploying Web servers and has evolved from handling basic file, print, and utility workloads to running mission-critical applications and databases, physically, virtually, and in the cloud. As Linux grows in importance in terms of value to the business, managing Linux environments to high standards of service quality — availability, security, and performance — becomes an essential requirement for business success.

Learn More

Sponsored by Red Hat

White Paper
Private PaaS for the Agile Enterprise

If you already use virtualized infrastructure, you are well on your way to leveraging the power of the cloud. Virtualization offers the promise of limitless resources, but how do you manage that scalability when your DevOps team doesn’t scale? In today’s hypercompetitive markets, fast results can make a difference between leading the pack vs. obsolescence. Organizations need more benefits from cloud computing than just raw resources. They need agility, flexibility, convenience, ROI, and control.

Stackato private Platform-as-a-Service technology from ActiveState extends your private cloud infrastructure by creating a private PaaS to provide on-demand availability, flexibility, control, and ultimately, faster time-to-market for your enterprise.

Learn More

Sponsored by ActiveState