PubSub-over-Webhooks with RabbitHub
This page is a mirrored copy of an article originally posted on the (now sadly defunct) LShift blog; see the archive index here.
Tue, 30 June 2009
RabbitHub is our implementation of PubSubHubBub, a straightforward pubsub layer on top of plain old HTTP POST — pubsub over Webhooks. It’s not well documented yet (understatement), but that will change.
It gives every AMQP exchange and queue hosted by a RabbitMQ broker a couple of URLs: one to use for delivering messages to the exchange or queue, and one to use to subscribe to messages forwarded on by the exchange or queue. You subscribe with a callback URL, so when messages arrive, RabbitHub POSTs them on to your callback. For example,
http://dev.rabbitmq.com/rabbithub/endpoint/x/amq.direct is the URL for delivering messages to the “amq.direct” exchange on our public test instance of RabbitMQ, and
http://dev.rabbitmq.com/rabbithub/subscribe/q/some_queue_name is the URL for subscribing to messages from the (hypothetical) queue “some_queue_name” on the broker.
(The symmetrical …/subscribe/x/… and …/endpoint/q/… also exist.)
The PubSubHubBub protocol specifies some RESTful(ish) operations for establishing subscriptions between message sources (a.k.a “topics”) and message sinks. RabbitHub implements these operations as well as a few more for RESTfully creating and deleting exchanges and queues.
Combining RabbitHub with the AMQP protocol implemented by RabbitMQ itself and with the other adapters and gateways that form part of the RabbitMQ universe lets you send messages across different kinds of message networks — for example, our public RabbitMQ instance, dev.rabbitmq.com, has RabbitHub running as well as the standard AMQP adapter, the rabbitmq-xmpp plugin, and a bunch of our other experimental stuff, so you can do things like this:
become XMPP friends with pshb@dev.rabbitmq.com (the XMPP adapter gives each exchange a JID of its own)
use PubSubHubBub to subscribe the sink http://dev.rabbitmq.com/rabbithub/endpoint/x/pshb to some PubSubHubBub source — perhaps one on the public Google PSHB instance. (Note how the given URL ends in “x/pshb”, meaning the “pshb” exchange — which lines up with the JID we just became XMPP friends with.)
wait for changes to be signalled by Google’s PSHB hub to RabbitHub
when they are, you get an XMPP IM from pshb@dev.rabbitmq.com with the Atom XML that the hub sent out as the body
RabbitHub is content-agnostic — you don’t have to send Atom around — so the fact that Atom appears is an artifact of what Google’s public PSHB instance is mailing out, rather than anything intrinsic in pubsub-over-webhooks.
We’ve also been experimenting with using http://www.reversehttp.net/ to run a PubSubHubBub endpoint in a webpage — see for instance http://www.reversehttp.net/demos/endpoint.html and its associated Javascript for a simple prototype of the idea. I’m playing with building a simple PSHB hub in Javascript using the same tools.
Comments
On 30 June, 2009 at 11:18 pm, wrote:
On 1 July, 2009 at 12:13 pm, wrote:
This article is a little short. I would have liked to see a little more explanation to fill things out a bit more.
I like that it is content agnostic.
On 1 July, 2009 at 7:48 pm, wrote:
[...] with RabbitHub July 1st, 2009 admin Leave a comment Go to comments PubSub-over-Webhooks with RabbitHub. RabbitMQ, the Erlang-powered AMQP message queue, is growing an HTTP interface based on webhooks [...]
On 3 July, 2009 at 12:14 am, wrote:
[...] PubSub-over-Webhooks with RabbitHub « LShift Ltd. RabbitHub is our implementation of PubSubHubBub, a straightforward pubsub layer on top of plain old HTTP POST — pubsub over Webhooks. It’s not well documented yet (understatement), but that will change. [...]
On 3 July, 2009 at 7:13 pm, wrote:
Tony,
Great write up. Does RabbitHub follow Google’s spec of PubSubHubbub draft 0.1? or its own derivative?
Thanks.
Peter
On 11 July, 2009 at 4:10 am, wrote:
@ryan: thanks! I’ll be writing more in future to help explain things better.
@peter: Yes, it follows the spec. It only implements the receiving and sending sides though: it doesn’t implement any of the Atom-specific parts, including the ping protocol. It also implements a couple of small extensions (undocumented, as yet, like the rest of the system :) ).
On 11 July, 2009 at 5:19 am, wrote:
I’ve updated the README at http://github.com/tonyg/rabbithub with the contents of this article as a start on some documentation; more to come.
On 10 August, 2009 at 1:43 am, wrote:
Great stuff!
I built an app to generate an atom feed, and figured out how to set up an exchange on dev.rabbitmq.com. The hardest part was figuring out how to subscribe using Google’s PubSubHubbub hub. I am happy to say that I can now get a XMPP message whenever I add an item to my feed!
Unfortunately, I seem to be getting some duplicates. I may have subscribed more than once…
[...] PubSubWebRabbitHubHook. Aside from the lack of HMAC authentication, I really like where PubSubHubBub is going, and the ease of letting RabbitMQ do the hard bits. [...]