One of the key features inside RabbitMQ is the ability to easily route messages between exchanges and queues, using any number of provided routing patterns. Out of the box, RabbitMQ provides a variety of algorithms to use when routing (based on the AMQP). You can route using the following algorithms:

  • Direct routing key matches (key="key")
  • Header based matching (headers["field"]="value")
  • Pattern based routing key matches (key="brown.owl", pattern="*.owl")

Between each of these, you gain the ability to cover a broad range of architectures and messaging patterns. The server plays a centric role in routing your messages; keeping routing logic outside of your applications. While this provides a good level of abstraction, sometimes you need more granular control over which messages go where.

Problem Space

For a moment, consider an architecture where your provider is sending messages to other clusters, rather than simply emitting messages into the ether for anyone listening to pick up. This is the use case I have been working with recently (albeit simplified), and is actually quite an interesting ask of RabbitMQ due to the abstraction of routing over to the server (and the AMQP). For the sake of example we'll assume that each message is to be routed to any number of clients, where the clients are provided by the publisher.

Topic Exchanges

After reading through the RabbitMQ blog posts about their improvements to topic routing it seemed that this use case was a good fit for topic exchange, and thinking about how this would work came out with a pretty trivial solution.

To understand how to work with topics, think of patterns as dot-separated keys where you can use the # character to mean zero or more keys and you can use the * character to mean exactly one key. You can also put arbitrary characters in to act as keys, but you cannot use * or # inside a key (this isn't Regex matching). For example, all of the following are valid patterns for the key a.b.c: 

a.b.c
a.*.c
a.b.*
*.b.*
*.*.c
*.*.*
#.a.#
#.b.*
#.b.c
#.c
#

Naturally this is extremely powerful and you can represent a wide variety of use cases using these patterns. For the client use case, you could use a key of <client>.<client>... and a pattern of #.<client>.# to listen to any messages which reference your client.

This works just as required, as it's essentially using a wildcard on both sides of the client identifier. However wildcards are typically a bad idea (especially in cases like this where there is a leading one), and this usage is no different. Binding in this way is extremely expensive and performance testing (with the rabbitmq-perf-test project) showed that the slowdown was too great for this to be a viable option.

Headers Exchanges

The other built-in alternative is the headers exchange, which also provides a functional solution (even if it is a little messy). Messages in RabbitMQ can have arbitrary headers attached to them, of various value types, and can be used for routing via the headers exchange. It essentially functions in the same way as a direct exchange, except that it treats each header pair as the routing match.

You can match on all headers, or a subset of the headers, or a mixture of both (via multiple bindings). For our use case, we can mock multiple routing keys using client identifiers as field names in the headers and give it any value (in our case 1) to test for presence. For simplicity, here is a representation of the above in JSON: 

{
  ...,
  "headers": {
    "client1": 1,
    "client2": 2,
    ...
  }
}

We can then bind to the exchange above using our client identifier and the match as arguments to the binding, in the form client1=1 to represent that we only want messages intended to be routed to client1. As with the topic exchange, functionally this is great but performance is again an issue. It appears that this exchange type simply exists to conform to AMQP and doesn't seem to be used frequently. Having said that, spending time reading through the source of the headers exchange seems to show that the matching algorithm is optimal.

Direct Exchanges

At this point you may have observed that, in essence, the client use case really only needs a direct exchange which supports multiple routing keys per message. Interestingly enough, the server implementation actually includes support in the models but it appears to either be unsupported in the transfer protocol, or just by the client implementations.

Fortunately for us, RabbitMQ supports extension via the use of plugins which allow you to define your own exchanges (amongst other things). Supporting this means that we can safely add a custom exchange which supports multiple routing keys during publish. Although we can create an exchange, we still have the limitation that the routing keys sent by clients are just Strings rather than lists.

One of the most common ways to represent multiple values in a String is to separate your values using a delimiter (for example's sake we'll use :). In our example use case, we could just list our clients using this delimiter in the form client1:client2:... and then route to each identifier in sequence. To make it more flexible, we can actually turn this into a self-describing implementation by requiring the first character in the key to be the delimiter (i.e. :client1:client2:...).

Rolling this into a plugin was easy enough as it can simply piggyback on the existing implementation backing the direct exchange (seeing as the internal models already contain lists of routing keys), and so that's how the x-delimiter-exchange came into being.

Delimiter Exchanges

The x-delimiter exchange is a custom exchange plugin for RabbitMQ which enables using a delimiter to provide multiple routing keys per message. It's backed by the same implementations backing the direct exchange, so it's very fast when compared to topic/header based solutions.

Running the x-delimiter exchange through the performance tests shows throughput basically indistinguishable to the results from the direct exchange. This is ideal because it means that we don't see any slowdown whilst also supporting the client identifier use case above. Below are some quick results from some of the performance tests executed when comparing the exchange types. 

DIRECT
    1x1
        id: test-200906-663, sending rate avg: 30233 msg/s
        id: test-200906-663, recving rate avg: 30231 msg/s
    1x4
        id: test-201026-084, sending rate avg: 12936 msg/s
        id: test-201026-084, recving rate avg: 51746 msg/s
    1x8
        id: test-201142-990, sending rate avg: 4146 msg/s
        id: test-201142-990, recving rate avg: 32150 msg/s
X-DELIMITER
    1x1
        id: test-102856-810, sending rate avg: 31285 msg/s
        id: test-102856-810, recving rate avg: 31283 msg/s
    1x4
        id: test-103247-860, sending rate avg: 13329 msg/s
        id: test-103247-860, recving rate avg: 53316 msg/s
    1x8
        id: test-103403-225, sending rate avg: 5141 msg/s
        id: test-103403-225, recving rate avg: 41063 msg/s

The PxC format above displays the number of producers against the number of consumers, and the delimiter exchange tests included two keys delimited by the : character. You can see that there's very little overhead against the default direct exchange, which is only logical since it basically just forwards to the direct exchange implementation under the hood. This also means that x-delimiter should be compatible with basically any RabbitMQ version since the behaviour of the direct exchange is defined by the AMQP specification.

This is the solution we're going to go for internally as it will scale as much as any of the other routable exchange, whilst also supporting the types of use cases we want to work with. You can find the exchange plugin on the GitHub repository whitfin/rabbitmq-delimiter-exchange along with instructions and releases on how to install it.