At the Forge - eBay Web Services

eBay's Web services API allows programs to search through on-line auctions, but only if the programmer doesn't get too frustrated first.

During the past few months, we have looked at the Web services offered by two of the largest companies on the Internet, Amazon and Google. Each of these companies has enormous databases at the core of their businesses. By opening up some of that data to the public via Web services APIs, they have made it possible for outside developers to create new and interesting applications. No longer must we write “screen scraper” programs that parse the HTML produced by Amazon or Google. Now, we can write a program that requests precisely the data we want and receives it in exactly the format we need.

Another major on-line contender is eBay, and its database of on-line sales might well be the largest ever assembled. eBay began solely as a site for on-line auctions, but it has moved far beyond that in recent years—with a fixed-price subsidiary (Half.com), fixed-price sales on the main eBay site (Buy it now) and third-party “stores” where people sell a variety of goods for a fixed or variable price.

For several years, eBay has run a developer program for programmers interested in tapping into its database. However, until recently, this developer program required that developers pay in order to participate. From a business perspective, it might have initially seemed foolish for eBay to give away access to its sales database, particularly when the developer program clearly costs money to set up and maintain. Whether it was due to pressure from Amazon and Google, or from individual developers, or if eBay simply decided that it would benefit from additional publicity and outside developers, eBay dropped fees—making it possible for everyone to try this service.

This month, we look at several aspects of the eBay Web services API. The API is too rich and extensive to discuss fully, so we look at the functionality that I believe most people will be interested in using—namely, that which lets you search through existing eBay auctions for items that are of interest. By the end of this article, you should understand how the API works, how to write programs that use REST to search through eBay's database and how to use that information for personal and business needs.

The idea behind Web services is quite simple. Instead of treating an HTTP transaction as a request for an HTML document, why not think of it as a remote procedure call? The HTTP request then becomes a method for invoking a procedure on a remote server, with the URL indicating which method should be invoked and the HTTP response containing the result of the call. In nearly all cases, the response is an XML document, allowing for the invoked procedure to return a complex data structure.

There are at least three different styles for invoking a Web service, and eBay supports all of them. SOAP is perhaps the most sophisticated method, using XML in both the request and the response, but it is also the most complex and the most likely to run into cross-platform incompatibilities. This is partly because SOAP has tried to standardize all of the possible method calls, data types and scenarios that might be needed—leading to a somewhat bloated specification and many places where vendors disagree on how best to adhere to the specification.

eBay also supports invoking Web services with what it calls the XML API. Because SOAP also consists of XML, I find this terminology to be a bit confusing, but Amazon also describes things in this way. So, until someone creates a useful acronym or name, we're stuck with it. APIs based on XML are basically stripped-down versions of SOAP, without much of the overhead associated with it, such as namespaces and highly specified methods to marshal complex data structures. eBay says it is possible to use either XML or SOAP to access the full functionality of its Web services.

If I had to choose between SOAP and XML, I usually would use XML. But eBay provides a third interface, which is more limited than the SOAP and XML APIs, but far easier to work with. This third option is known as REST (short for representational state transfer), and anyone familiar with URLs should immediately understand how it works. The parameters are passed in the URL, using the standard name=value syntax. A REST invocation thus looks like http://www.example.com/method?param1=value1&param2=value2.

REST invocations are useful only for searching through eBay's catalogs. If you want to monitor sales, adjust your shopping basket, add listings to your store or even send messages to sellers and buyers, you must use the XML or SOAP API. The size of the API documentation says it all: eBay's REST documentation is 29 pages long, and documentation for the SOAP and XML APIs is more than 1,600 pages long in each case.

Because the application we are building is supposed to search only through existing offers, rather than add a new item for sale, we can get away with using the REST API. The REST API makes it easier to jump right in, and it provides all of the functionality with less programming overhead.