Match response handler with request in VertX

Question

Let's say I have a Load Balancer (LB) in front of 1..n VertX (V) instances, each VertX instance is connected to a queue (Q), and I have 1..m Backends (BE).

A user clicks on a button which makes a post request or even opens a web socket, the load balancer forwards the request to one of the VertX instances, which fires a request to the queue, one of the Backends consumes the message and sends a response back; if the correct VertX instance consumes it, it can lookup the response handler and write a response to the user, if the wrong VertX instance consumes it, there won't be a response handler to write a response to and the user will wait indefinitely for a response.

See this sketch:

Alternatively, V2 dies and the load balancer reconnects the user to V1 which means even if I could send it back to the exact same one that made the request, it's not guaranteed to still be there once the response comes back, but the user might still be there awaiting a response via another VertX instance.

What I'm currently doing is to generate a GUID for each new connection, then as soon as the websocket connects, store the websocket handler inside a hashmap against the GUID and then when the BE wants to respond, it does a fanout to all 1..n VertX instances, the one that currently has the correct GUID in its hashmap can then write a response to the user. Same for handling POST / GET in this manner.

Pseudocode:

queue.handler { q ->

  q.handler {
    val handler = someMap.get(q.guid)
    // only respond if handler exists
    if (handler != null){
        handler.writeResponse(someresponsemessagehere)
    }
  }

}

vertx.createHttpServer().websocketHandler { ws ->
  val guid = generateGUID()
  someMap.put(guid, ws)                                                                    
  ws.writeFinalTextFrame("guid=${guid}")
  ws.handler { 
    val guid = extractGuid(it)
    // send request to BE including generated GUID
    sendMessageToBE(guid, "blahblah")
  }

}.requestHandler { router.accept(it) }.listen(port)

This does however mean that if I have a 1000 VertX applications running, that the backend will need to fanout its message to a 1000 frontend instances of which only one will make use of the message.

VertX seems like it already takes care of async operations very well, is there a way in VertX to identify each websocket connection instead of having to maintain a map of GUIDs mapped to websocket handlers / post handlers?

Also, referring to the picture, is there a way for V3 to consume the message, but still be able to write a response back to the websocket handler that's currently connected to V2?

I'm not familiar with VertX so I may be missing something that otherwise would be obvious but why not use 1..n queues (one for each VertX (V) instance) or use a queue that supports selectors so that clients can consume only messages for them (e.g. activemq.apache.org/how-do-i-consume-a-specific-message.html)? — mfulton26
We're using RabbitMQ, I don't think RabbitMQ can do that - in other words, specifically consume specific messages from a single queue. In order to get routing to a 1000 queues, you'll need to setup a topic exchange, a topic exchange is limited to 255 topics if I remember correctly. Actually I was looking for a similar feature in RabbitMQ 2 years ago: stackoverflow.com/questions/25489301/… If we were using ActiveMQ, maybe that would have been a solution. — Jan Vladimir Mostert
@JanVladimirMostert As a side question, why do you need the queue(s) in your "formula"? Vert.x is asynchronous by default, and assuming all your operations are non-blocking and asynchronous, I think you don't really need the queue...and the "back-ends" can be also verticles (of any type). — x80486
@ɐuıɥɔɐɯ It's a polyglot system, Java, Kotlin, C#, VB, perl, Python, PHP and even some bash scripts and there's close to 200 applications sitting on the queues already happily chatting to each other - I'm converting them from RPC style (blocking) to async. For most of the applications, going async is easy, for frontend applications or integrations, this is tricky since there's a physical connection that waits for a response to be written to it, so it needs to be processed by the application that's keeping the connection open. — Jan Vladimir Mostert

Alexey Soshin Alexey Soshin · Accepted Answer · 2016-12-19T18:33:18

What you're missing from your diagram is the Vertx EventBus.

Basically you can assume that your V1...Vn are interconnected:

V1<->V2<->...<->Vn

Let's assume that Va receives your outbound Q message (the red line), that is intended for Vb.
It then should send it to Vb using EventBus:

eventBus.send("Vb UUID", "Message for Vb, also containing WebSocket UUID", ar -> {
  if (ar.succeeded()) {
    // All went well
  }
  else {
    // Vb died or has other problems. Your choice how to handle this
  }
});

Match response handler with request in VertX

1 Answers