ZMQ - request/response with multiple front service instances and backend workers

1
votes

Here is what I am trying to achieve -

A REST service (deployed on multiple instances) gets a request, processes it and passes it on to a fleet of worker instances.

The worker does the bigger chunk of work, prepares the response and sends it back.

The main service in the mean time has finished some more work related to the original request, merges that with the worker response and sends it back to caller on the request connection.

How can I model this through ZMQ?

I have tried doing this - - create a PUSH socket and do a "bind" in the service instance. - workers "connect" to the corresponding PULL socket. - Since the same service instance has to get the response and no other service instance, with every message sent to the PUSH socket, service appends a response queue name. - Worker receives the message, does the work and pushes (does a connect) on the response queue given in the message. - Service does a bind on it's response queue and when it receives the response, does the rest of the work and responds to the service caller.

The problem - - All threads in a service instance has to do some synchronization to get hold of the PUSH queue to send a request to worker. - With heavy load this single PUSH queue starts choking. - When I deploy this service code on multiple instances and bind to the same socket, things break.

Is there a standard approach for achieving this with ZMQ? The reason we decided to use ZMQ is less connection management as is needed if worker was another HTTP service (retries, disconnections, connection pools etc) and better throughput with ZMQ.

1
javamultithreadingsocketszeromqjzmq
I found this and another solution with a streamer device. But both of these approaches involve a broker instance running in the middle and becomes the single point of failure. Is there a version without that limitation? – anindyaju99
I'll try to write something later involving ROUTERs and DEALERs and such, but two things jump out. Have you read zguide.zeromq.org, and threads in the service instance should be communicating with each other through ZMQ sockets, not through synchronization. – dsolimano
@dsolimano can't say I recall everything I read in that guide, but surely I went through that. In my case communication across threads is not the problem. – anindyaju99
"The problem - - All threads in a service instance has to do some synchronization to get hold of the PUSH queue to send a request to worker" - they don't, you shouldn't be sharing the socket between threads. Hmm. Let me see if I can find an example. – dsolimano

1 Answers

0
votes

Ok, here's what I would do, as a high level sketch. I'm sure it's not quite complete, and there's a bit of frame management that needs to be done int the various processes.

We have three types of processes total in my vision. The first are your server processes with worker threads, and the second are your set of worker processes. This is very similar to the "Worked Example: Inter-Broker Routing" in the ZeroMQ guide. I'll throw in an intermediate message broker, the third process, to connect the servers to the workers, so you can add servers without having to reconfigure the clients, and vice versa.

Each server has a ROUTER socket to talk to service threads, serviced by a thread we'll call the router thread.

The service threads use REQ sockets to talk to the router thread. When the service thread gets a request, it sends a request to the workers by sending a message to the REQ socket. It then does its own work. When that work is done, it tries to read from the REQ socket, which will block until the response is received. When it gets the response, it combines everything and replies to its client.

The router thread has the ROUTER socket to talk to its peers, and a DEALER socket to talk to the message broker.

Now onto the broker process, which is basically the example process. It's got two ROUTER sockets, one to talk to the service processes (call it the frontend), and one to talk to the worker processes (call it the backend). It polls for work on the frontend ROUTER, and when it receives some, it polls for a request on the backend ROUTER, which indicates a worker process is free, and then forwards the request on.

The worker processes use the REQ mechanism in an odd fashion. Each has a REQ socket, that they use to request work from the broker process. They get a response consisting of work, and they do the work and then send another REQ consisting of the results and a request for more work.

One thing to note about this is that nowhere do we use locking or synchronization in Java - instead everything is handled via message passing, using ZeroMQ as the transport.

Outline