Helena Edelson

Akka and Spring Committer, Scala, FP, Cloud Computing, Data Mining, Machine Learning, Distributed Architecture, API Design

  • cloud

  • RSS Subscribe

  • Topics

  • Archives



    • @helenaedelson on Twitter
    • Helena Edelson on LinkedIn
    • GitHub Octocat

      helena @ github

      • Status updating...


Archive for the 'Cloud' Category

Orderly Load-Time Provisioning And Graceful Shutdown of Akka Nodes

Posted by Helena Edelson on 13th August 2013

Simple (and truncated) example of the CloudExtension’s load-time ordered provisioning and ordered graceful shutdown. Unfortunately this had to be written in an older version of scala and akka – for now. MyNodeGuardian.scala is started in CloudExtension.register() and is a sample of using ProvisioningGuardian which extends OrderedGracefulShutdown: View Gist

Share/Save

Posted in Akka, Cloud, Scala, Scalability | Comments Off

TopAByBJob: MapReduce Written in Scala With Scalding, Pailfiles, for S3 data

Posted by Helena Edelson on 11th August 2013

My latest Gist: TopAByBJob is a MapReduce problem written in Scala with Scalding, Pailfiles, for S3 data https://gist.github.com/helena/6194946

Share/Save

Posted in Big Data, Cloud, MapReduce, S3, Scala, Scalding | Comments Off

Recent Gists on Akka Cluster Metrics-Aware Adaptive Load-Balancing Routers

Posted by Helena Edelson on 11th May 2013

These are some of the Gists I wrote for The New Cluster Metrics-Aware Adaptive Load-Balancing Routers and The Cluster Metrics API for letitcrash.com which I recycled for my last Scala user group presentation on Akka

Share/Save

Posted in Akka, Big Data, Cloud, Concurrency, Scala, Scalability | Comments Off

Closing Over An Akka Actor Sender In The Receive

Posted by Helena Edelson on 24th March 2013

Accidental closing over a sender is easy to get wrong but mayhem if you do it. The rule is simply never close over the sender method in a block of code that is potentially executed in another thread, such as a scheduled task or a Future. The trick is to capture the current sender in a val, as illustrated below:

Unsafe


Here are 3 examples of what not to do. It all depends on what is actually done in the receive. For example, if Futures or other threads are involved and a call to forward or sender is executed in callbacks (such as Future.onSuccess) then you have a problem:

1. Calling sender on another thread than the original message in Actor.receive

1
2
3
4
5
6
def receive = {
  case SomeMessage =>
    context.system.scheduleOnce(5 seconds) {
      sender ! DelayedResponse  // Danger!
    }
}

Here the thing to watch out for is { sender ! DelayedResponse } because it is run on another thread than the original message. The sender is associated with current message being processed, and in the Scheduler case illustrated above, several other messages might have been processed during the 5.seconds before the call to the method sender, such that the correct sender is no longer associated with the message that was processed when the scheduled task was triggered.

2. Shared mutable state

1
2
3
4
5
6
7
8
class MyActor extends Actor {
    var state = State()
    def receive = {
      case _ =>
        Future { state = newState }
        otherActor ? message onSuccess { r => state = r }
    }
}

The above code will break your application in weird ways. Because the sender changes for every message this will cause a shared mutable state bug.

3. Calling sender in a Future during Actor.receive could return a null or be the wrong ActorRef

1
2
3
4
5
class MyActor extends Actor {
    def receive = {
      case _ =>  Future { expensiveCalculation(sender) } // painful, and you owe me a beer if you do it
    }
}

The return value of the sender method after processing the current message might be null until the next message is received, but that doesn’t matter. Simply use a val if you close over sender in a a block of code that might be executed by another thread.

Safe


Here are a few examples that are safe, the first because we simply freeze the sender reference:

1
2
3
4
5
6
7
8
9
class MyActor extends Actor {
    def receive = {
        case SomeMessage =>
            val requestor = sender
            context.system.scheduleOnce(5 seconds) {
              requestor ! DelayedResponse
           }
   }
}

Or better yet, keep a clean Actor.receive all together – a best practice

1
2
3
4
5
6
7
8
class MyActor extends Actor {
    def receive = {
        case msg: SomeMessage => receiveSomeMessage(sender, msg)
   }
  def receiveSomeMessage(requestor: ActorRef, msg: SomeMessage): Unit = context.system.scheduleOnce(5 seconds) {
        requestor ! DelayedResponse
   }
}

It is safe to close over self and a thread-safe ActorRef

1
2
3
4
5
class MyActor extends Actor {
    def receive = {
      case _ => Future { expensiveCalculation() } onComplete { f => self ! f.value.get }
    }
}

It is safe to close over a fixed value and a thread-safe ActorRef

1
2
3
4
5
6
7
class MyActor extends Actor {
    def receive = {
      case _ =>
        val currentSender = sender // freeze the sender to a val
        Future { expensiveCalculation(currentSender) }
    }
}

Find Out More

Some examples were taken from “Akka Concurrency” by Derek Wyatt, others from the Akka Docs.

Share/Save

Posted in Akka, Cloud, Concurrency, Java, Scala | No Comments »

Routing Topologies for Performance and Scalability with RabbitMQ

Posted by Helena Edelson on 1st August 2011

Designing a good routing topology for a highly-scalable system can be like mapping a graph. Many things need to be considered, for instance the problem, constraints of the environment, those of the messaging implementation, and performance strategies. What we often run up against is a lack of flexibility and expressivity in fitting routing to our needs. Here is where RabbitMQ stands out.

Basic Concepts

Anyone familiar with messaging in general knows the concept of routing messages from A to B. Routing can be simplistic or quite complex, and when designing a routing topology for a scalable, complex system it must be elegant. Kept clean and decoupled, components can throttle nicely with varying loads. This can be expressed as a simple map or complex graph. In its simplest form a routing topology can be expressed as nodes, for instance hierarchical nodes:

For those new to RabbitMQ or AMQP (note that Rabbit works with many protocols including STOMP, HTTP, HTTPS, XMPP, and SMTP), here are some basic component descriptions:

  • Exchange The entity within the server which receives messages from producer applications and optionally routes these to message queues within the server
  • Exchange type The algorithm and implementation of a particular model of exchange. In contrast to the “exchange instance”, which is the entity that receives and routes messages within the server
  • Message queue A named entity that holds messages and forwards them to consumer applications
  • Binding An entity that creates a relationship between a message queue and an exchange
  • Routing key A virtual address that an exchange may use to decide how to route a specific message

For point-to-point routing, the routing key is usually the name of a message queue. For topic pub-sub routing the routing key is usually hierarchical in nature:

api.agents.agent-{id}.operations.{operationName}

In more complex cases the routing key may be combined with routing on message header fields and/or its content. An exchange examines a message’s properties, header fields, body content, and possibly data from other sources, then decides how to route the message. A binding pattern derived from the above routing key idea might look like api.agents.*.operations.* where we bind exchange E1 to queue Q1 with binding pattern api.agents.*.operations.* so that any messages sent to E1 route to Q1 if their routing key matches the binding pattern.

A Rabbit broker is structured differently than a JMS Broker. Each RabbitMQ server is comprised of at least one node (broker), or more typically, nodes in a cluster. Each node has a default virtual host, “/”, and further virtual hosts can be created such as “/develoment”. Rabbit virtual hosts are like Tomcat virtual hosts and partition broker data into sub-sets. Within these virtual hosts are exchanges and queues. When a user connects with its credentials, it is connecting to a virtual host on a Rabbit node.

Here we connect to a Rabbit node, declare an exchange to publish to, a queue to consume from, a binding pattern, then publish a few messages, using the RabbitMQ java client api:

package org.demo.simple.rabbit;

import com.rabbitmq.client.Channel;
import com.rabbitmq.client.Connection;
import com.rabbitmq.client.ConnectionFactory;
import com.rabbitmq.client.QueueingConsumer;
import java.io.IOException;
import java.util.ArrayList;
import java.util.List;

public final class RocketSender {

 public void sendRockets() throws IOException {
     List<String> rocketsWithRoutings = new RocketRouter().build();

     Connection connection = new ConnectionFactory().newConnection();
     Channel channel = connection.createChannel();

     String rocketExchange = "rockets.launched";
     channel.exchangeDeclare(rocketExchange, "topic");
     String rocketQueue = channel.queueDeclare().getQueue();
     channel.queueBind(rocketQueue, rocketExchange, "galaxies.*.planets.*");

     for (String rocketTo : rocketsWithRoutings) {
         channel.basicPublish(rocketExchange, "galaxies.*.planets." + rocketTo, null, rocketTo.getBytes());
     }

     channel.close();
     connection.close();
 }
}

A simple consume of “landed” rockets could look like:

QueueingConsumer queueingConsumer = new QueueingConsumer(channel);
channel.basicConsume(rocketQueue, false, queueingConsumer);

int landed = 0;
while (landed < launched) {
    QueueingConsumer.Delivery delivery = queueingConsumer.nextDelivery();
    channel.basicAck(delivery.getEnvelope().getDeliveryTag(), false);
    String rocketLanded = new String(delivery.getBody());

    if (rocketLanded.equalsIgnoreCase("Alderaan")) {
        System.out.println("That's no moon, that's a space station.");
    }
    landed++;
}

The Problem

In considering what routing strategies perform best in scalable environments where performance itself can also be improved, there many options. One of the great things about messaging in general is the variety of configurations available, and figuring out the right ones that solve both current and growing requirements.

To keep things simple let's consider two strategies:

  1. Highly-partitioned routing with hierarchical routing keys, fewer topic exchanges
  2. A larger number of direct exchanges and queues with far less routing partitions

Each scenario follows this use case: each application that must scale is both producer and consumer:

Producer Consumer

Where To Start

It is a good idea to take stock of your environment and its components before delving into a routing solution that will scale cleanly and efficiently over time. For example, what lends to scaling? Generally, decoupling, distribution, asynchrony, parallelism, levels of abstraction and indirection to name a few. Then consider what elements are current or potential bottlenecks. It is a basic principle that high traffic/volume pathways require more efficient throughput or you incur risk of bottlenecks in your distribution. One exercise is to rank these in terms of traffic or as a heat map. Next, can you classify your traffic – are there overarching patterns, topics or similar message types, and what are the relationships? Now start to consider consolidation, how and where might efficiency be improved, and apply tested patterns that resolve those heat points, decouple for scale, and increase performance.

General Routing Considerations

All exchange types behave differently. Here are a few general rules:

  • If you have a finite domain of routing keys in an application's graph then many fanout exchanges might be the right fit  (1:1 mapping of exchange per routing  key)
  • If you have a potentially infinite number of routing keys, consider topic exchanges
  • For topic routing, performance decreases as the number of bindings increase
  • Fanout exchanges are very fast because they have no routing to process yet if bound to a large number of queues that changes
  • Direct exchanges are a faster form of topic exchanges, provided you do not need the wild card
  • Troubleshooting problems across 100,000+ queues could be tedious versus a topology with more bindings, fewer exchanges and queues
  • A very high number of exchanges and queues take up more memory which may be significant but this really depends

As of RabbitMQ 2.4.0, released March 23, 2011, a new topic routing algorithm optimization is available that is 60 times faster at peak than the previous topic algorithm. Due to this, one recommendation is to go for less exchanges and queues, and more routing because the time increase is now minimal.

Performance

What is Cheap?

In terms of memory cost, exchanges and bindings. In Erlang, which RabbitMQ is built on, each node (broker) is a process, as is each queue. By default the Erlang VM process limit is set to 1M, which can be raised. However, an exchange is not a process for scalability reasons, it is simply a row in RabbitMQ's built-in Mnesia database. In a cluster, declaring an exchange causes it to appear on all nodes of the cluster, while declaring a queue creates it on only one of the nodes. This explains why exchanges survive node restarts or creating a node in a cluster, yet queues do not.

Be wary of binding churn. In strategy number 2, if you create many new queues and their bindings, whenever consumers attach you might run into problems. For instance, given exchanges E1...En to which many messages are being published, whenever consumer Cm connects, it creates bindings from its own queue to all of E1...En which may cause problems, depending on the rate of connections.

To alleviate binding churn, consider exchange-to-exchange bindings, new as of version 2.3.1. Each consumer could have its own secondary exchange Ym which must not be auto-delete. Then bind all of E1...En to Ym. In this way these bindings always exist. In this scenario, whenever consumer Cm connects it simply has to declare its queue and bind that queue to Ym. If Ym is a fanout exchange, it will be very fast and reduce the binding churn rate to 1 per connection, rather than potentially n per connection.

Exchange-to-Exchange Binding

Use Cases

Exchange-to-Exchange Scalable Use Case

Consider a server application with autonomous agents. Each agent is on a virtual machine that is part of an elastically-scaled system. As each agent starts up it sends a message to the server that it is online, followed by many other messages such authentication and data transfer. If we have 1,000 agents, each declaring 50 direct exchanges, queues and bindings, then each agent must know the server's queues in order to fulfill the binding contract on queue.declare operations. That is not a scalable solution.

Now consider creating shared topic exchanges: one exchange for the agent to server pathway,  another for the server to agent pathway, and a third to handle unauthenticated agents which routes only to those queues that do not require security. Now we partition with binding patterns, message routing keys and bring one set of those up for each server to be shared by all agents that connect to it. Then, in its simplest form, as each agent comes online it declares a private exchange and queue, and binds its exchange to the shared topic exchanges.

Our relationships are now expressed by exchange-to-exchange mappings which reduces churn rate and decouples agents from having to 'know' the server queues. Using this pattern the system is  clean, decoupled, and scalable.

Basic Routing Topology

Elastic-Scaling Use Case

Let's take the previous scenario a step further. We are already using topic pub-sub routing over scenario 2: many direct routings. Now let's say the system requirement bumps up to scale clusters of our server application in a data center with 50,000 or more agents. How can we throttle varying loads?

The authenticated client exchange routes messages from agent to server. It handles all operations publishing messages to singular-consumer queues, including those producing the highest frequency of messages. This is a potential bottleneck under the current topology with roughly 60,000 messages per minute for 10,000 clients or 86,400,000 messages per day. This is easily resolvable, RabbitMQ can handle over 1 billion messages per day depending on your configuration, for example whether or not you are persisting messages.

Our server applications are running a RabbitMQ cluster. Remember that in a cluster, declaring an exchange causes it to appear on all nodes, while declaring a queue creates it on only one of the nodes, so we have to configure a solution.

Load-Balancing Between Producers and Consumers

To efficiently handle these potentially very high loads as more client applications (Agents) come online, we can modify this topology in several ways. First, an optimization from the above configuration to load-balance messages across a Rabbit cluster. We can create one queue for each node in the Rabbit cluster. If we have four nodes, for each high-traffic queue we create hfq.{0,1,2,3} for that operation. Now each agent can randomly pick a node by a number between zero and three, or a more sophisticated round-robin implementation, to publish to. With RabbitMQ there are RPC calls or you can use the Rabbit management plugin to GET the number of nodes, which you can use in your round-robin algorithm.

Worker Queues With Round-Robin Dispatching

Worker Queues, or Task Queues, are generally used to efficiently distribute time-consuming tasks among multiple workers and easily parallelise work. In addition, this topology applies to eliminating the need to execute resource-intensive tasks and having to block until they complete. Running several worker queues allows these tasks to be distributed among them.

AMQP Work Queues, Round Robin Load Balancing

With Work Queues, by default, Rabbit uses a round-robin distribution method, sending each message to the next consumer in sequence. Each consumer receives roughly the same number of messages. If you declare a queue and spin up 3 competing consumers, bind them to the exchange, and send 20,000 messages, messages zero will route to the first consumer, message one to the second, message two to the third and so on. If we begin building up a backlog of tasks, we can simply add more workers, allowing the system to scale easily.

Performance

Memory

Neither option above will necessarily induce high load in RabbitMQ. There are no hard limits to the number of exchanges and queues, one can create, and running 100,000 queues on one broker is fine. With the right tuning and enough RAM you can run well over a million queues.

RabbitMQ dynamically pushes messages to disk to free up RAM, thus the memory footprint of a queue is not dependent on its contents. After a queue is idle for 10 seconds or more it will "hibernate" which causes GC on that queue. As a result, the amount of memory a queue requires can dramatically shrink. For example, it might be possible for 1000 empty, idle queues to take up 10MB of RAM. When they're all active (even if empty), they might of course, depending on memory fragmentation, consume much more memory. Forcing them back into hibernation to test behavior is difficult because the Erlang VM does not hand back memory to the OS immediately.

You can, however, observe a large process that hibernates and has very fragmented memory because the amount reclaimed can be sufficient to force the VM to hand back memory to the OS. If you run a test that steadily increases the memory footprint of Rabbit you could observe the effect of hibernation on idle processes as it reduces the rate of increase of memory use.

Erlang is a multi-threaded VM which takes advantage of multiple cores. It presents green threads to the developer which are called 'processes' because unlike threads, they conceptually do not share an address space.

Transactions

Transactions on 10,000 messages can take as along as four minutes to publish. A new RabbitMQ feature called Publisher Confirms is more than 100 times faster than the same, but transactional, code. If you are not explicitly required to implement transactions but do need the verification you might consider this option.

The Take-Away

Here are some final takeaways to help you squeeze the greatest performance gains out of your implementation:

  • The new topic routing algorithm optimization is 60 times faster at peak
  • Topic binding patterns using wildcards '*', which matches a single word, is much faster than '#', which matches zero or more words. Wildcard '#' takes longer to process in the routing table than '*'
  • Exchange-to-exchange bindings improve decoupling, increase topology flexibility, reduce binding churn, and help increase performance
  • Exchanges and bindings are very light weight
  • RabbitMQ Publisher Confirms are more than 100 times faster than AMQP transactions
  • After a queue is idle for >=10 seconds it will "hibernate", inducing GC on the queue, resulting in a dramatic decrease in memory required for that queue
  • Worker Queues help parallelize and distribute workloads
  • Distributing worker queues in the Rabbit cluster helps scale
  • Load-balance your topology

This is by no means a thesis on the subject, there are indeed many more patterns, topologies and performance details to consider. A strategy, as always, depends on so many factors but I hope this encapsulates enough to help or at least get one thinking in the right directions.

Get It

RabbitMQ Source on GitHub
RabbitMQ Binary Downloads and Plugins
Erlang Downloads
Spring AMQP API for RabbitMQ in Java and .NET
Hyperic to monitor RabbitMQ
Maven


<dependency>
<groupId>com.rabbitmq</groupId>
<artifactId>amqp-client</artifactId>
<version>${rabbitmq.version}</version>
<exclusions>
<exclusion>
<groupId>commons-cli</groupId>
<artifactId>commons-cli</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>commons-io</groupId>
<artifactId>commons-io</artifactId>
</dependency>

Share/Save

Posted in AMQP, Broker Topology, Cloud, Erlang, Messaging, Open Source, Performance, RabbitMQ, Scalability | No Comments »

Routing, Performance and Scalability with RabbitMQ

Posted by Helena Edelson on 7th April 2011

My latest post on the Spring Team blog
Routing Topologies for Performance and Scalability with RabbitMQ

Share/Save

Posted in AMQP, Cloud, Erlang, JMS, Messaging, Open Source, Performance, RabbitMQ, Scalability | No Comments »

Why RabbitMQ for Cloud?

Posted by Helena Edelson on 2nd February 2011

We know that Messaging itself affords moving from a SQL -> Database -> File System model and into the Network layer itself with:

  • Logical decoupling
  • Physical decoupling
  • Temporal decoupling

but I’ve been asked why I would choose implementing RabbitMQ in a cloud solution. This is a big topic, and there is a great deal more I’d say conversationally but for a relatively quick overview let’s start from the ground up.

Why AMQP?

AMQP is an open, wire-level binary protocol for Message Orientated Middleware that was created by and continues to developed collaboratively for users in mission-critical real-world scenarios. AMQP solutions are deployed to high performance clustering and grid computing problems and is developed for typical business scenarios:

1. distributes data to hundreds of clients in real time across a global network
2. accepts workloads to process from those clients and handles them reliably in a clustered transaction processing engine
4. connects to other partners over the Internet to exchange data
5. carries out all these functions in a secure manner which would pass an audit
6. monitors application traffic to build up a real-time activity

AMQP implementations offer fire and forget where you can absolutely trust that your messages will reach their destination, extremely fast delivery of data to a large number of consumers (publish sub-event notification), and
secure, firewall friendly file transfer.

“AMQP is a good fit for cloud MaaS and other wide-area, Internet-based messaging purposes, because it defines the wire protocol and communication semantics without restricting the details of the messaging system implementation or the API. AMQP’s goal is to enable interoperability among disparate vendors’ platforms.”
- Roy Shulte (Gartner), Review of AMQP and other protocol options.

More: amqp.org

Why Erlang?

All major implementations of AMQP such as Open AMQP, Qpid and RedHat MRG are built on a foundational technology. All but one are built on C++. RabbitMQ is built on Erlang. Erlang insures concise binary matching, has built-in support for concurrency, distribution and fault tolerance, has a built-in distributed database, is open source, and operates on many platforms. Erlang is also a runtime environment, similar to the java virtual machine, allowing code compiled on one architecture to run anywhere. The runtime system also allows code in a running system to be updated without interrupting the program.

More: erlang.org

Why RabbitMQ?

Many large-scale projects are using RabbitMQ for complex, scalable, cloud topologies, such as Observatories Initiative (OOI), NASA Nebula, Hiroku, Unique Identification Authority of India (200 million messages per day peak), and CERN because

  • It’s wicked fast and reliable
  • Supports security and transactions, persistence, client acks, and flow control
  • Widely proven and used heavily on Amazon EC2
  • It is open source with support services and several very active user lists
  • Hugely interoperable
    • RabbitMQ brokers run on many platforms and offer several clients (Java, .net and others), all which speak AMQP protocol to each other. Anyone can write a RabbitMQ client in any language: Ruby, Java, Spring, Python, C, C#, Perl, Erlang, Lisp, Haskell, PHP, _your_language_here.
  • Easily clusterable
  • Works with several protocols: AMQP, STOMP, XMPP, 0MQ
  • Highly-flexible topologies
  • Can work with Pacemaker and associated tools to provide High Availability
  • Supports several extension points via plug-ins, e.g. exchange types and backing queue storage
  • Provides management and monitoring of brokers
    • vFabric Hyperic
    • RabbitMQ command line tool and plugin for real-time activity and management
  • Messaging services hosted in the cloud with RabbitMQ as part of overall cloud infrastructure services are highly-supported

An open source RabbitMQ solution stack might look like this: Spring, Tomcat, Hibernate, MySQL, Hadoop, MapReduce, Chef, Apache httpd, RabbitMQ.

More: rabbitmq.com

Resource List And Use Cases

Share/Save

Posted in AMQP, Cloud, Cloud Ops, Concurrency, Erlang, Messaging, Open Source, RabbitMQ | No Comments »

Open Artificial Intelligence for the Cloud – Initial Diagram

Posted by Helena Edelson on 20th September 2010

This is a working (initial) idea of where an AI appliance might fit into a virtualized, elastic environment. I will update this concept soon as I created it early this summer and have many changes to now make to it. The parent post: Open Source Artificial Intelligence for the Cloud

© Copyright – not for re-use.

Share/Save

Posted in AI, Artificial Intelligence, Cloud, Cloud Ops, Messaging, Open Source, RabbitMQ, Spring | No Comments »

Open Source Artificial Intelligence for the Cloud

Posted by Helena Edelson on 18th September 2010

The idea of introducing intelligent computing so that software can make decisions autonomously and in real time is not new but it is relatively new to the cloud. Microsoft seems to be the strongest force in this realm currently but this is not open source. The idea of clouds that manage themselves steps way beyond Cloud Ops as we know it, and is the only logical, and necessary, next step.

I’ve been researching various open source AI platforms and learning algorithms to start building a prototype for cloud-based learning and action for massive distributed Cloud Ops systems and applications. Once could off er an AI cloud appliance eventually but I will start with a prototype and build on that using RabbitMQ (an AMQP messaging implementation), CEP, Spring, Java as a base. I’ve been looking into OpenStack, the open source cloud computing software being developed by NASA, RackSpace, and many others. Here is the future GitHub repo: http://github.com/helena/OCAI.

Generally to do these sort of projects you need a large amount of funding and engineers with PhD’s, none of which I have. So my only alternative is to see how we might take this concept and create a light weight open source solution.

Imagine having some of these (this being not a comprehensive list) available to your systems:

AI Technology

  • Planning and Scheduling
  • Heuristic Search
  • Temporal Reasoning
  • Learning Models
  • Intelligent Virtual Agents
  • Autonomic and Adaptive Clustering of Distributed Agents
  • Clustering Autonomous Entities
  • Constraint Predictability and Resolution
  • Automated Intelligent Provisioning
  • Real-time learning
  • Real-world action
  • Uncertainty and Probabilistic Reasoning
  • Decision Making
  • Knowledge Representation

I will go into these topics further in future posts.

About me: I am currently an engineer at SpringSource/VMware on the vFabric Cloud Application Platform side of things, however this project is wholly outside of that, I feel the need to pursue something I’m interested in. I work with the RabbitMQ team and a member of the AMQP Working Group.

Share/Save

Posted in AI, AMQP, Artificial Intelligence, CEP, Cloud, Cloud Ops, Java, Messaging, Open Source, RabbitMQ, Software Development, Spring | No Comments »

RabbitMQ and Erlang Intallation for Mac OSX

Posted by Helena Edelson on 11th June 2010

I’ve been a linux developer for years so I was all thumbs having to look up everything when setting up a new dev environment on a mac. Here’s some tips for setting up a local RabbitMQ server to play with (i.e. not using macports locally, not the EC2 install etc):

Requirements

  1. Apple XCode to compile Erlang with gcc. The XCode download could not take longer. Start it, go build an enterprise app in ROO, deploy it to the cloud. When you come back it may have completed.
  2. wget from http://ftp.gnu.org/pub/gnu/wget
  1. tar -xzf wget-{version}.tar.gz
  2. cd wget{version}
  3. ./configure
  4. make
  5. sudo make install

Install and build Erlang

First, Check for the latest Erlang GA Release and note the version.

  1. cd to the dir you wish to install into
  2. wget http://erlang.org/download/otp_src_{version}.tar.gz
  3. tar xzvf otp_src_{version}.tar.gz
  4. cd otp_src_{version}
  5. NOTE: run ./configure –help to see what opts you want to run with
  6. ./configure
  7. make
  8. sudo make install

Install RabbitMQ Server

First check the latest version of RabbitMQ Server and note the filename.

  1. wget http://www.rabbitmq.com/releases/rabbitmq-server/v2.3.1/rabbitmq-server-generic-unix-{version}.tar.gz
  2. tar xzvf rabbitmq-server-generic-unix-{version}.tar.gz
  3. Complete the install and config from http://www.rabbitmq.com/install.html#generic-unix, making sure the logs, clustering, data and config dirs are set up
  4. Start RabbitMQ: run rabbit../sbin/rabbitmq-server

Infinitely easier on Ubuntu.

Share/Save

Posted in Cloud, Erlang, Messaging, RabbitMQ, Spring ROO | 3 Comments »

 
a2a_init("page"); a2a_init("page"); a2a_init("page"); a2a_init("page"); a2a_init("page"); a2a_init("page"); a2a_init("page"); a2a_init("page"); a2a_init("page");