Advanced Message Queuing Protocol (AMQP)

AMQP is an open standard for enterprise messaging, designed to support messaging for almost any distributed or business application.

What if, using a single service call, you easily could ask a computing cloud to give you the readings from thermometers in 100 different locations? Or, perhaps you'd like to know the status of the 89 servers under your control. In the past, you might have accomplished those things by writing a server dæmon. Your dæmon might have managed each of hundreds of connections, conducting specific operations on each connection. However, with the advent of AMQP and the Apache Qpid Project, it's possible to concentrate on the data processing and let another program handle the messaging.

AMQP is an innovative open messaging protocol. Created by John O'Hara and others at JPMorgan to replace proprietary products, the AMQ protocol defines both the wire-level formats and the behavior of messaging server and client software. Using the above example, you could send a single message to the AMQP server with a topic such as server_stats or thermometer_readings. The AMQP server listens for messages with those topics and routes the messages to the applications connected to the AMQP server.

A Bit of History

AMQP began in 2003 with John O'Hara at JPMorgan-Chase. O'Hara was looking for a messaging solution that provided high durability, extremely high volume and a high degree of interoperability. In the types of environments addressed with AMQP, there is an economic impact if a message is lost, arrives late or is processed improperly. With volumes greater than 500,000 messages per second, the requirements were high. The commercial products that were available at the time could not deliver the level of service required, and banks were known to develop their own enterprise middleware to fill in the gaps. However, developing enterprise middleware is complex and difficult, and bank middleware would come and go.

As he reflected on other highly successful protocols, such as Ethernet, TCP/IP and HTTP, O'Hara noted several similarities. Namely, each protocol was royalty-free and not encumbered by patents. Furthermore, the protocols had a strong specification created by an independent body. Freely available implementations of the protocol specifications allowed developers to pick them up and find interesting uses for them quickly. Strong governance and user-driven design made these protocols a technical and economic success.

With AMQP, O'Hara wanted to have a freely available implementation of the AMQ protocol in use in a mission-critical place at JPMorgan. With this goal in mind, he contracted with the iMatix Corporation to create the first implementation, OpenAMQ. This implementation then was put into production in a trading application with more than 2,000 users.

Today, many companies collaborate on AMQ. Several brokers are available, including RabbitMQ, OpenAMQ and Apache Qpid (also known as Red Hat MRG Messaging). In this article, I describe the Apache Qpid server. Up for discussion is the Qpid M4 release, and you can download it via the link in the Resources for this article. I also demonstrate how to compile and install the C++ version of the server and write example applications in Python.

Anatomy of a Server

Figure 1 depicts the anatomy of a Qpid server. It is important to know about three components of an AMQP server: local queues, server queues and exchanges.

Figure 1. Anatomy of a Qpid Server Work Flow

The exchange determines message delivery based on the message header. Exchanges can provide different delivery schemes, such as direct (deliver this message to queue XYZ), publish-subscribe (deliver this message to all queues subscribed to topic spring.flowers) and XML (all messages that match XPath query Z go to Queue Y). A server queue is a queue that resides on the server and receives messages from the exchange. A local queue is a queue associated with an instance of an application. Local queues are bound to server queues, so any message delivered to the server queue appears on the local queue. More than one local queue can be bound to a server queue. This is handy when you have a farm of machines (or processes) responding to requests. In this case, messages will be delivered from the server queue to the local queue on a round-robin basis.

In addition to the server terms, note that programs reading from queues are called consumers, and those writing to exchanges are producers. This can become confusing when you have applications that act as both consumers and producers. Therefore, it makes sense to use standard terminology for clients and servers, where a client sends a request and expects a response on a reply queue, and a server listens for messages and responds as requested.

The Qpid broker discussed here comes with XML files describing the AMQP specification. These files define the formats used by the server and clients. The server and libraries use these specifications to formalize parameters, such as wire format, server commands and error messages. Managing these specifications outside the server allows you to maintain compatibility across different server vendors and different server versions. In theory, you should be able to replace a Qpid server with OpenAMQ, RabbitMQ or any other AMQP-compliant server and have it work out of the box. In practice, different servers support different versions of the specification or require different options. For example, the Qpid Java Client supports three versions of the protocol: 0-8, 0-9 and 0-10. However, the C++ client supports only 0-10 in its latest release. RabbitMQ, a competing AMQP broker, supports only 0-8 and 0-9 of the specification. Because of this, the best results are when using clients and brokers from the same product line.

Apache Qpid and its commercial counterpart, Red Hat MRG Messaging, are versatile products. They offer many features not covered here. For example, you can use SSL and InfiniBand fabric as interconnects, and you can control how clients connect to your server via ACLs and authentication. I highly recommend the Red Hat MRG documentation for further reference on these features.

______________________

Comments

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

Article Graphic

Josh Kramer's picture

Hello Everyone,

Please check out my blog update for the correct graphic:

http://www.globalherald.net/jb01/weblog/22.html

Thanks!
-JK

White Paper
Linux Management with Red Hat Satellite: Measuring Business Impact and ROI

Linux has become a key foundation for supporting today's rapidly growing IT environments. Linux is being used to deploy business applications and databases, trading on its reputation as a low-cost operating environment. For many IT organizations, Linux is a mainstay for deploying Web servers and has evolved from handling basic file, print, and utility workloads to running mission-critical applications and databases, physically, virtually, and in the cloud. As Linux grows in importance in terms of value to the business, managing Linux environments to high standards of service quality — availability, security, and performance — becomes an essential requirement for business success.

Learn More

Sponsored by Red Hat

White Paper
Private PaaS for the Agile Enterprise

If you already use virtualized infrastructure, you are well on your way to leveraging the power of the cloud. Virtualization offers the promise of limitless resources, but how do you manage that scalability when your DevOps team doesn’t scale? In today’s hypercompetitive markets, fast results can make a difference between leading the pack vs. obsolescence. Organizations need more benefits from cloud computing than just raw resources. They need agility, flexibility, convenience, ROI, and control.

Stackato private Platform-as-a-Service technology from ActiveState extends your private cloud infrastructure by creating a private PaaS to provide on-demand availability, flexibility, control, and ultimately, faster time-to-market for your enterprise.

Learn More

Sponsored by ActiveState