How to build rabbit management console

Using RabbitMQ (and other AMQP brokers) one can configure queues, exchanges and bindings during runtime via protocol.  The benefit of configuring via protocol is that the client does not have to rely on internal broker’s implementation.  As long as the client is capable of packing the packets and dropping them onto the network, it is ready to play.  However, there is more information to be desired via protocol than just the construction of necessary moving parts that makes the client and broker happy.  Not to worry, the rabbitmqctl command line utility comes to the rescue.  It provides enough meta information to monitor and interact with the broker.  rabbitmqctl is shell script that delegates to rabbit_control.erl to communicate with various Erlang nodes in order to extract the information.

In the last few days I have been working on a web management console for RabbitMQ.  I basically need to emulate all the features provided by rabbitmqctl, but I don’t want to shell out to the command line.  So I have two choices: build the web console in Erlang or talk to Erlang nodes via JInterface¹.  I chose the latter.  JInterface ships with Erlang Otp, so if you already have RabbitMQ running you should be able to locate the OtpErlang.jar some where on your system.

To extract the information via JInterface is mostly painless, but tedious.  I’ll provide basic instructions here and maybe someone will find this information helpful.  I’ll assume the basics of Java programming are known.

The basic program structure looks like the following:

OtpSelf self = new OtpSelf(”monitor”, “YOUR-ERLANG-COOKIE-VALUE-HERE”);
OtpPeer peer = new OtpPeer(”rabbit@server”);
// Be sure to close the connection!
OtpConnection conn = self.connect(peer);
OtpErlangList arguments = new OtpErlangList();
conn.sendRPC(module, function, arguments);
OtpErlangObject result = conn.receiveRPC();

OtpSelf creates a node from Java that has the capability to interact with other Erlang nodes.  The first argument is the name of the node which can be called anything.  The second argument is the most important.  In order for Erlang nodes to communicate with each other, all the nodes must use the same magic cookie.  If the cookie value isn’t supplied properly, you will receive an OtpAuthException when trying to connect.  The RabbitMQ cookie file on my Macbook is located in /opt/local/var/lib/rabbitmq/.erlang.cookie.  Depending on your platform it might be stored in a different location.  I actually ran into a couple problems with the Erlang cookie file, but I’ll discuss it at the end of the article.

OtpPeer is the rabbit node you are connecting to.  Replace the “server” with the host name (hostname -s) of your machine that is running RabbitMQ.

In order for RabbitMQ to return any information, the correct function within a module must be invoked.  For example if you want to list all the virtual hosts in a broker:

conn.sendRPC(”rabbit_access_control”, “list_vhosts”, new OtpErlangList());

So how did I know it’s in module “rabbit_access_control” under a function with no arguments called “list_vhosts”?  Remember I said in the beginning that rabbitmqctl does this already?  If you look in rabbit_control.erl you can figure out pretty much anything you need to know.  Search for list_vhosts within the file and you will come across the following code:

action(list_vhosts, Node, [], Inform) ->
Inform(”Listing vhosts”, []),
display_list(call(Node, {rabbit_access_control, list_vhosts, []}));

The second parameter of the call function is what one is after.  The first value of the tuple (surrounded by curly braces) is the module name followed by the function and arguments.  Going back to the list_vhosts example, let see what receivedRPC returns when requesting all the virtual hosts.  The easiest way is just output the toString on the OtpErlangObject.

System.out.println(result); -> [#Bin<1>]

As one can see in this example the result is actually a collection of Erlang binary (#Bin) objects.  Since there’s only one virtual host on my broker, only one binary value is returned.  Let’s extract the value of the binary.  We’ll cheat since we know there are only one virtual host in this case.

OtpErlangBinary binary = (OtpErlangBinary)((OtpErlangList)result).elementAt(0);
System.out.println(new String(binary.binaryValue()));

The code above is making a lot of assumptions.  I did this on purpose in order to focus on the important concepts, so don’t write the code like that in a production system.  Everything should be pretty self explanatory, maybe except for why I wrapped binaryValue with a new String.  binaryValue returns a byte array, passing it to a new String will convert it to human readable format.  Pretty easy right?

I have shown the simplest case, but for the more complicated return values all the concepts remain the same.  Remember I said it gets pretty tedious?  Most of the time it doesn’t just return a binary value in a collection, but it’s usually wrapped in a tuple.  It gets worse, sometimes tuples are wrapped in tuples and so on.  Like I said before, it’s not hard, just time consuming.

On my Macbook a ran into a couple issues with Erlang cookie file.  For whatever reason I had to copy the .erlang.cookie into my home directory.  I also had to map my hostname (hostname -s) to 127.0.0.1 in my /etc/hosts file.  Before attempting writing any monitoring code, make sure you can actually connect to the Erlang instance cleanly.  Once the connection is made, make sure you close the connection, otherwise you won’t be able to make subsequent connections without bouncing the server.

I have successfully emulated all the features of rabbitmqctl in Clojure.  I plan to throw JRuby on top of it for the frontend whenever I get a chance.

All these tedious parsing may go away after the management capability is a first class citizen in the AMQP protocol.  Until then, this is what you have to do if you want to monitor RabbitMQ.

If you get stuck, the Java doc for JInterface is pretty helpful.  I can illustrate a more complicated example in detail if someone finds this information helpful.  Please feel free to drop me a comment or email.

---

1. This is not entirely true.  I actually came across another option I didn’t know about when I wrote this post.  Apparently you can also use Py-Interface as express by Dmitriy Samovskiy on the RabbitMQ mailing list to accomplish the same thing.  The knowledge expressed here is general enough and still applies even if you use something else like Py-Interface.  If you know another way, please let me know.