Zato—Agile ESB, SOA, REST and Cloud Integrations in Python

Laying Out the Services

The first thing you need is to diagram the integration process, pull out the services that will be implemented and document their purpose. If you need a hand with it, Zato offers its own API's documentation as an example of how a service should be documented (see https://zato.io/docs/progguide/documenting.html and https://zato.io/docs/public-api/intro.html):

  • Zato's scheduler is configured to invoke a service (update-cache) refreshing the cache once in an hour.

  • update-cache, by default, fetches the XML for the current month, but it can be configured to grab data for any date. This allows for reuse of the service in other contexts.

  • Client applications use either JSON or simple XML to request long-term rates (get-rate), and responses are produced based on data cached in Redis, making them super-fast. A single SIO Zato service can produce responses in JSON, XML or SOAP. Indeed, the same service can be exposed independently in completely different channels, such as HTTP or AMQP, each using different security definitions and not interrupting the message flow of other channels.

Figure 1. Overall Business Process

Implementation

The full code for both services is available as a gist on GitHub, and only the most interesting parts are discussed.

linuxjournal.update-cache

Steps the service performs are:

  • Connect to treasury.gov.

  • Download the big XML.

  • Find interesting elements containing the business data.

  • Store it all in Redis cache.

Key fragments of the service are presented below.

When using Zato services, you are never required to hard-code network addresses. A service shields such information and uses human-defined names, such as "treasury.gov"; during runtime, these resolve into a set of concrete connection parameters. This works for HTTP and any other protocol supported by Zato. You also can update a connection definition on the fly without touching the code of the service and without any restarts:


 1 # Fetch connection by its name
 2 out = self.outgoing.plain_http.get('treasury.gov')
 3
 4 # Build a query string the backend data source expects
 5 query_string = {
 6  '$filter':'month(QUOTE_DATE) eq {} and year(QUOTE_DATE) eq
{}'.format(month, year)
 7 }
 8
 9 # Invoke the backend with query string, fetch 
   # the response as a UTF-8 string
10 # and turn it into an XML object
11 response = out.conn.get(self.cid, query_string)

lxml is a very good Python library for XML processing and is used in the example to issue XPath queries against the complex document returned:


1 xml = etree.fromstring(response)
2
3 # Look up all XML elements needed (date and rate) using XPath
4 elements = xml.xpath('//m:properties/d:*/text()', 
  ↪namespaces=NAMESPACES)

For each element returned by the back-end service, you create an entry in the Redis cache in the format specified by REDIS_KEY_PATTERN—for instance, linuxjournal:rates:2013:09:03 with a value of 1.22:


 1 for date, rate in elements:
 2
 3   # Create a date object out of string
 4   date = parse(date)
 5
 6   # Build a key for Redis and store the data under it
 7   key = REDIS_KEY_PATTERN.format(
 8       date.year, str(date.month).zfill(2), 
         ↪str(date.day).zfill(2))
 9   self.kvdb.conn.set(key, rate)
10
12   # Leave a trace of our activity
13   self.logger.info('Key %s set to %s', key, rate)

linuxjournal.get-rate

Now that a service for updating the cache is ready, the one to return the data is so simple yet powerful that it can be reproduced in its entirety:


 1 class GetRate(Service):
 2 """ Returns the real long-term rate for a given date
 3 (defaults to today if no date is given).
 4 """
 5 class SimpleIO:
 6     input_optional = ('year', 'month', 'day')
 7     output_optional = ('rate',)
 8
 9 def handle(self):
10     # Get date needed either from input or current day
11     year, month, day = get_date(self.request.input)
12
13     # Build the key the data is cached under
14     key = REDIS_KEY_PATTERN.format(year, month, day)
15
16     # Assign the result from cache directly to response
17     self.response.payload.rate = self.kvdb.conn.get(key)

A couple points to note:

  • SimpleIO was used—this is a declarative syntax for expressing simple documents that can be serialized to JSON or XML in the current Zato version, with more to come in future releases.

  • Nowhere in the service did you have to mention JSON, XML or even HTTP at all. It's all working on a high level of Python objects without specifying any output format or transport method.

This is the Zato way. It promotes reusability, which is valuable because a generic and interesting service, such as returning interest rates, is bound to be desirable in situations that cannot be predicted.

As an author of a service, you are not forced into committing to a particular format. Those are configuration details that can be taken care of through a variety of means, including a GUI that Zato provides. A single service can be exposed simultaneously through multiple access channels each using a different data format, security definition or rate limit independently of any other.

______________________

Dariusz Suchojad is a systems architect specializing in SOA/ESB/EAI/BPM/SSO with 12 years of experience completing projects for enterprise customers in telecommunications and banking.

Webinar
One Click, Universal Protection: Implementing Centralized Security Policies on Linux Systems

As Linux continues to play an ever increasing role in corporate data centers and institutions, ensuring the integrity and protection of these systems must be a priority. With 60% of the world's websites and an increasing share of organization's mission-critical workloads running on Linux, failing to stop malware and other advanced threats on Linux can increasingly impact an organization's reputation and bottom line.

Learn More

Sponsored by Bit9

Webinar
Linux Backup and Recovery Webinar

Most companies incorporate backup procedures for critical data, which can be restored quickly if a loss occurs. However, fewer companies are prepared for catastrophic system failures, in which they lose all data, the entire operating system, applications, settings, patches and more, reducing their system(s) to “bare metal.” After all, before data can be restored to a system, there must be a system to restore it to.

In this one hour webinar, learn how to enhance your existing backup strategies for better disaster recovery preparedness using Storix System Backup Administrator (SBAdmin), a highly flexible bare-metal recovery solution for UNIX and Linux systems.

Learn More

Sponsored by Storix