We know that Messaging itself affords moving from a SQL -> Database -> File System model and into the Network layer itself with:
- Logical decoupling
- Physical decoupling
- Temporal decoupling
but I’ve been asked why I would choose implementing RabbitMQ in a cloud solution. This is a big topic, and there is a great deal more I’d say conversationally but for a relatively quick overview let’s start from the ground up.
AMQP is an open, wire-level binary protocol for Message Orientated Middleware that was created by and continues to developed collaboratively for users in mission-critical real-world scenarios. AMQP solutions are deployed to high performance clustering and grid computing problems and is developed for typical business scenarios:
1. distributes data to hundreds of clients in real time across a global network
2. accepts workloads to process from those clients and handles them reliably in a clustered transaction processing engine
4. connects to other partners over the Internet to exchange data
5. carries out all these functions in a secure manner which would pass an audit
6. monitors application traffic to build up a real-time activity
AMQP implementations offer fire and forget where you can absolutely trust that your messages will reach their destination, extremely fast delivery of data to a large number of consumers (publish sub-event notification), and
secure, firewall friendly file transfer.
“AMQP is a good fit for cloud MaaS and other wide-area, Internet-based messaging purposes, because it defines the wire protocol and communication semantics without restricting the details of the messaging system implementation or the API. AMQP’s goal is to enable interoperability among disparate vendors’ platforms.”
- Roy Shulte (Gartner), Review of AMQP and other protocol options.
All major implementations of AMQP such as Open AMQP, Qpid and RedHat MRG are built on a foundational technology. All but one are built on C++. RabbitMQ is built on Erlang. Erlang insures concise binary matching, has built-in support for concurrency, distribution and fault tolerance, has a built-in distributed database, is open source, and operates on many platforms. Erlang is also a runtime environment, similar to the java virtual machine, allowing code compiled on one architecture to run anywhere. The runtime system also allows code in a running system to be updated without interrupting the program.
Many large-scale projects are using RabbitMQ for complex, scalable, cloud topologies, such as Observatories Initiative (OOI), NASA Nebula, Hiroku, Unique Identification Authority of India (200 million messages per day peak), and CERN because
- It’s wicked fast and reliable
- Supports security and transactions, persistence, client acks, and flow control
- Widely proven and used heavily on Amazon EC2
- It is open source with support services and several very active user lists
- Hugely interoperable
- RabbitMQ brokers run on many platforms and offer several clients (Java, .net and others), all which speak AMQP protocol to each other. Anyone can write a RabbitMQ client in any language: Ruby, Java, Spring, Python, C, C#, Perl, Erlang, Lisp, Haskell, PHP, _your_language_here.
- Easily clusterable
- Works with several protocols: AMQP, STOMP, XMPP, 0MQ
- Highly-flexible topologies
- Can work with Pacemaker and associated tools to provide High Availability
- Supports several extension points via plug-ins, e.g. exchange types and backing queue storage
- Provides management and monitoring of brokers
- vFabric Hyperic
- RabbitMQ command line tool and plugin for real-time activity and management
- Messaging services hosted in the cloud with RabbitMQ as part of overall cloud infrastructure services are highly-supported
An open source RabbitMQ solution stack might look like this: Spring, Tomcat, Hibernate, MySQL, Hadoop, MapReduce, Chef, Apache httpd, RabbitMQ.
Resource List And Use Cases
- Messaging in the Cloud – AMQP, RabbitMQ and Spring
- Cloud And Grid Exchange 2010 – RabbitMQ Cloud Messaging Use Cases
- RabbitMQ: Messaging in the Cloud – Matthew’s presentation to VMworld 2010 that covers an introduction to messaging with common use cases, an AMQP 101 and an overview of RabbitMQ including its features and application to cloud messaging.
- Use Cases For Cloud Messaging