XMLTP/L, XMLTP Light

Describing how XMLTP Light works and the environments for which it is suited.

XMLTP/L, or XMLTP Light, is a lightweight RPC protocol that uses XML to encode the stream of data. XMLTP/L has been designed to do fast RPC calls over an intranet, within an enterprise. More specifically, the first purpose of XMLTP/L is to forward transactions (RPCs) to a database server. But, it also can be used to do method calls to any server that follows the common RPC technique introduced by XML-RPC and older client/server protocols.

The origins of XMLTP/L come from the inherent limitations associated with proprietary client/server protocols: basically, they are proprietary. XMLTP/L is distributed as source code and is licensed under the GNU Library General Public License (GNU LGPL).

Because of the languages used in the current implementation of XMLTP/L, developers who use C, Python and Java should find this article more interesting. But, we expect ports will be made to other scripting and web application languages, such as PHP and Ruby.

The name XMLTP Light evokes the design goals and purposes of this protocol:

  • transport TP Light (see Note 1) remote procedure calls (RPC);

  • use an XML syntax to allow compatibility with various tools, extant or future;

  • have good performance, with modest requirements (therefore, real practical scalability using simple hardware configurations); and

  • have a lightweight and robust implementation.

To achieve its goal of being fast and lightweight, XMLTP/L does not try to do everything and acknowledges the following constraints and limitations as non-goals:

  • it uses a subset of XML (a full-fledge XML parser can parse XMLTP/L, but not the reverse);

  • it is not a universal transport for all datatypes;

  • it is not the most standardized or the most buzzwords-compliant technology to appear now, yesterday or tomorrow; and

  • it is not as easy to install as XML-RPC (which is less than 20KB of code in the Python 2.x standard installation).

XMLTP/L is much more boring than other XML protocols, such as SOAP or XML-RPC, or any newer web services protocols that will appear real-soon-now. But, if it fills someone's need for a non-proprietary, robust and flexible RPC protocol on their intranet, then that would be great.

XMLTP/L should be appealing to people who use stored procedure calls and want more flexibility (like integrating multiple DBMS or building a custom non-SQL dataserver). It works for people who need some level of XML compliance without suffering response time greater than 300 or 500 ms, as is the norm for all the implementations of RPC-over-XML that I have tried so far. I have not tried all of them but enough to convince me to write a custom XML subset parser in Bison and C.

Figure 1. XMLTP/L Server Interconnections

Figure 1 shows a schema of XMLTP/L server inteconnection. Only one dataserver is shown, but it is possible to put two or many behind the RRGX, a gateway that does RPC Routing (more about RRGX below). Also in Figure 1, you can see a small box called XML-RPC or SOAP bridge. This piece does not exist yet. But, it would be rather easy to create such a bridge, as the initial specs for XMLTP/L are similar to those for XML-RPC.

Technical Specifications

The current implementation of XMLTP/L uses plain TCP/IP socket for transport, and the connections are persistent. In other words, once a socket connection is established, it is used for many RPC calls. This situation is different from XML-RPC, where a new HTTP connection (a new TCP/IP connection) typically is made for each RPC call. This is one of the reasons why XMLTP/L is faster than XML-RPC.

Furthermore, the RRGX gateway and the Java classes (RPC call) use connection pools. This technique is another boost for performance when many threads of an application server make many RPC calls.

The lightness of the implementation is a third factor for good speed. XMLTP/L is a simple XML-based protocol, and it supports only four datatypes in the RPC parameters and in the tabular result set(s), which can be returned by the stored procedures. These datatypes are:

  • integer: 32 bits, most likely (Python allows for larger *int*, but XML2SYB uses ANSI "C" 32 bits *int* on GNU/Linux on x386)

  • float: double precision, most likely

  • timestamp: "YYYY-MM-DD HH:MM:SS.mmm"

  • string: max of 255 characters.

For any of these datatypes, a value can be flagged as *Null*. In this case, the data associated with that value should be sent empty by the server, and it should be ignored by the client.

If we look at all the details, a XMLTP/L response contains a return status (always), one or many result set(s) (optional), a messages(s) (optional) and an output parameter(s) (optional).

The XML syntax of both the RPC call and the response in XMLTP/L is lighter than the equivalent in XML-RPC. Here is an exampe of a RPC call in XMLTP/L:

<?xml version="1.0" encoding="ISO-8859-1" ?>
<procCall<proc>RPC_PROC_1</proc>
 <param>
  <name>@out_param</name
  <int>-9999</int>
  <attr>114</attr>
 </param>
 <param>
  <name>@param_1</name>
  <str>adenosine</str>
  <attr>0012</attr>
 </param>
</procCall>

Here's the same RPC call, using the XML-RPC syntax:

<?xml version='1.0'?>
<methodCall><methodName>RPC_PROC_1</methodName>
 <params>
  <param>
  <value>array><data>
   <value><string>@out_param</string></value>
   <value><int>-9999</int></value>
   <value><string>114</string></value>
  </data></array></value>
  </param>
  <param>
  <value><array><data>
   <value><string>@param_1</string></value>
   <value><string>adenosine</string></value>
   <value><string>0012</string></value>
  </data></array></value>
  </param>
 </params>
</methodCall>

The difference in the byte count is even more dramatic when the RPC call has more parameters and also in a response with a large number of rows.

A main design goal of XMLTP/L was to pump a high volume of transactions in and out of a relational database. We wanted some level of vendor independance, specifically so users would suffer less from the vendors' frequent upgrades to their native client-server APIs. Also, we wanted to be able to mix datasources and perform RPC routing in a way that would be invisible to the web application server or to other XMLTP/L clients connected to the RRGX.

XMLTP/L acts like a universal database RPC protocol. The whole intranet can use XMLTP/L for RPC calls, and vendors' proprietary protocols can be isolated on a server near the database itself. The RRGX (RPC Router Gateway for XMLTP/L) can integrate multiple database servers together as if if it was a single server. Be aware that XMLTP/L does not automate synchronisation of transactions on multiple dataservers; each RPC is independent of all other calls.

The RRGX does not use any proprietary client/server APIs. This RRGX allows calling procedures in multiple database servers as it routes the RPCs to the various converter programs, according to the names of the procedures. Those routing rules are defined in the configuration file of the RRGX.

The RRGX needs one converter per database protocol. Such converter programs are external programs built from the vendor's API libraries and the XMLTP/L modules. Currently, XML2SYB (XML to Sybase) is already implemented.

It also is possible to code stored procedures in Python in a server similar to the RRGX, which is derived from the generic gxserver.py module. Many other small features are in the RRGX that make it useful in a real production environment:

  • log of events and messages (error, warning, trace & debug)

  • dynamically adjustable trace level

  • built-in operator commands, such as ps, who, stats and so on

  • gates that can be closed while maintenance is done on the database(s) behind the gateway:

    • RPC gate (global)

    • connection pools gates

    • connection gate (global)

    • application specific gates (one gate for a list of RPC names).

    When a gate is closed, a user-defined message can be sent back to the clients.

  • RPC result interceptions (with passive queueing or active re-forwarding)

______________________

Comments

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

Re: Slow XMLRPC implementations

Anonymous's picture

Regarding the comments about slow XMLRPC calls: there is no reason the same approach (lightweight XML parser, persistent socket communications, say via HTTP/1.1) shouldn't provide the same response times within standard XMLRPC that you are getting with this system.

Check out XmlRpc++ for an example: http://xmlrpcpp.sourceforge.net

Re: Slow XMLRPC implementations

jftouchette's picture

Your point is valid, but, there are other features lacking from XML-RPC:

no login authentication (no "session"), no attribute (size) for a string

parameter or column... At some point, hacking/tuning an existing protocol until

it becomes incompatible with its existing implementations is a worse option

than creating yet another new "RPC" protocol. One should remember that

XMLTP/L first objective is to do fast transaction forwarding over a _LAN_

and that it has no ambition to replace SOAP or XML-RPC, which are

promoted for "web services" usage.

White Paper
Linux Management with Red Hat Satellite: Measuring Business Impact and ROI

Linux has become a key foundation for supporting today's rapidly growing IT environments. Linux is being used to deploy business applications and databases, trading on its reputation as a low-cost operating environment. For many IT organizations, Linux is a mainstay for deploying Web servers and has evolved from handling basic file, print, and utility workloads to running mission-critical applications and databases, physically, virtually, and in the cloud. As Linux grows in importance in terms of value to the business, managing Linux environments to high standards of service quality — availability, security, and performance — becomes an essential requirement for business success.

Learn More

Sponsored by Red Hat

White Paper
Private PaaS for the Agile Enterprise

If you already use virtualized infrastructure, you are well on your way to leveraging the power of the cloud. Virtualization offers the promise of limitless resources, but how do you manage that scalability when your DevOps team doesn’t scale? In today’s hypercompetitive markets, fast results can make a difference between leading the pack vs. obsolescence. Organizations need more benefits from cloud computing than just raw resources. They need agility, flexibility, convenience, ROI, and control.

Stackato private Platform-as-a-Service technology from ActiveState extends your private cloud infrastructure by creating a private PaaS to provide on-demand availability, flexibility, control, and ultimately, faster time-to-market for your enterprise.

Learn More

Sponsored by ActiveState