Zato—Agile ESB, SOA, REST and Cloud Integrations in Python

Laying Out the Services

The first thing you need is to diagram the integration process, pull out the services that will be implemented and document their purpose. If you need a hand with it, Zato offers its own API's documentation as an example of how a service should be documented (see https://zato.io/docs/progguide/documenting.html and https://zato.io/docs/public-api/intro.html):

  • Zato's scheduler is configured to invoke a service (update-cache) refreshing the cache once in an hour.

  • update-cache, by default, fetches the XML for the current month, but it can be configured to grab data for any date. This allows for reuse of the service in other contexts.

  • Client applications use either JSON or simple XML to request long-term rates (get-rate), and responses are produced based on data cached in Redis, making them super-fast. A single SIO Zato service can produce responses in JSON, XML or SOAP. Indeed, the same service can be exposed independently in completely different channels, such as HTTP or AMQP, each using different security definitions and not interrupting the message flow of other channels.

Figure 1. Overall Business Process

Implementation

The full code for both services is available as a gist on GitHub, and only the most interesting parts are discussed.

linuxjournal.update-cache

Steps the service performs are:

  • Connect to treasury.gov.

  • Download the big XML.

  • Find interesting elements containing the business data.

  • Store it all in Redis cache.

Key fragments of the service are presented below.

When using Zato services, you are never required to hard-code network addresses. A service shields such information and uses human-defined names, such as "treasury.gov"; during runtime, these resolve into a set of concrete connection parameters. This works for HTTP and any other protocol supported by Zato. You also can update a connection definition on the fly without touching the code of the service and without any restarts:


 1 # Fetch connection by its name
 2 out = self.outgoing.plain_http.get('treasury.gov')
 3
 4 # Build a query string the backend data source expects
 5 query_string = {
 6  '$filter':'month(QUOTE_DATE) eq {} and year(QUOTE_DATE) eq
{}'.format(month, year)
 7 }
 8
 9 # Invoke the backend with query string, fetch 
   # the response as a UTF-8 string
10 # and turn it into an XML object
11 response = out.conn.get(self.cid, query_string)

lxml is a very good Python library for XML processing and is used in the example to issue XPath queries against the complex document returned:


1 xml = etree.fromstring(response)
2
3 # Look up all XML elements needed (date and rate) using XPath
4 elements = xml.xpath('//m:properties/d:*/text()', 
  ↪namespaces=NAMESPACES)

For each element returned by the back-end service, you create an entry in the Redis cache in the format specified by REDIS_KEY_PATTERN—for instance, linuxjournal:rates:2013:09:03 with a value of 1.22:


 1 for date, rate in elements:
 2
 3   # Create a date object out of string
 4   date = parse(date)
 5
 6   # Build a key for Redis and store the data under it
 7   key = REDIS_KEY_PATTERN.format(
 8       date.year, str(date.month).zfill(2), 
         ↪str(date.day).zfill(2))
 9   self.kvdb.conn.set(key, rate)
10
12   # Leave a trace of our activity
13   self.logger.info('Key %s set to %s', key, rate)

linuxjournal.get-rate

Now that a service for updating the cache is ready, the one to return the data is so simple yet powerful that it can be reproduced in its entirety:


 1 class GetRate(Service):
 2 """ Returns the real long-term rate for a given date
 3 (defaults to today if no date is given).
 4 """
 5 class SimpleIO:
 6     input_optional = ('year', 'month', 'day')
 7     output_optional = ('rate',)
 8
 9 def handle(self):
10     # Get date needed either from input or current day
11     year, month, day = get_date(self.request.input)
12
13     # Build the key the data is cached under
14     key = REDIS_KEY_PATTERN.format(year, month, day)
15
16     # Assign the result from cache directly to response
17     self.response.payload.rate = self.kvdb.conn.get(key)

A couple points to note:

  • SimpleIO was used—this is a declarative syntax for expressing simple documents that can be serialized to JSON or XML in the current Zato version, with more to come in future releases.

  • Nowhere in the service did you have to mention JSON, XML or even HTTP at all. It's all working on a high level of Python objects without specifying any output format or transport method.

This is the Zato way. It promotes reusability, which is valuable because a generic and interesting service, such as returning interest rates, is bound to be desirable in situations that cannot be predicted.

As an author of a service, you are not forced into committing to a particular format. Those are configuration details that can be taken care of through a variety of means, including a GUI that Zato provides. A single service can be exposed simultaneously through multiple access channels each using a different data format, security definition or rate limit independently of any other.

______________________

Dariusz Suchojad is a systems architect specializing in SOA/ESB/EAI/BPM/SSO with 12 years of experience completing projects for enterprise customers in telecommunications and banking.

Geek Guide
The DevOps Toolbox

Tools and Technologies for Scale and Reliability
by Linux Journal Editor Bill Childers

Get your free copy today

Sponsored by IBM

Webcast
8 Signs You're Beyond Cron

Scheduling Crontabs With an Enterprise Scheduler
On Demand
Moderated by Linux Journal Contributor Mike Diehl

Sign up now

Sponsored by Skybot