1
votes

I'm using reactphp/zmq.

How can I have multiple push workers within multiple pull workers, is that possible?

A only can have multi pull and single push, as in README's example:

$push->connect()
$pull->bind()

Or single pull and multi pushes:

$push->bind()
$pull->connect()

When I try to set both as connect the pulls doesn't receive the messages.

Otherwise, trying to start more than one process with bind it throws:

ZMQSocketException: Failed to bind the ZMQ: Address in use

Should I have a middleware? ????

(5555)                      (5556)
push -|  (5555) > (5556)   |- pull
push -|-> pull  &  push <- |- pull
push -|   bind  /  bind    |- pull
connect                    connect
2
What are you solving with that proposed architecture? Why would you need to have multiple pushers? You can't have multiple pushers and then have random amount of workers that pull. You need to have a single endpoint where all workers PUSH and a single endpoint where all workers PULL from. Luckily, ZeroMQ supports TIPC protocol that makes this easy, but it's probably wiser that you explain what exactly you're trying to do so we can suggest optimal architecture.Mjh
I'm building a email validation tool where when a new list is added, it starts a new pusher that adds its emails to the queue while multiple pullers will be listening to that queue to validating them.Leo Cavalcante
You have the fan-out (pusher, the one that gives the task) and you have the worker, the one that performs the task. You don't have the queue. ZeroMQ is transport layer, not queue (it uses queue internally though). What you need is a service in the middle that acts as a queue - it receives and sends tasks. Your pusher pushes to queue, your pull-er pulls from the queue. That service would use pull to receive the task and push to hand it out to workers. You use push to give it tasks and pull to take tasks from it.Mjh
Ok, so in the end this middleware thing I'm thinking is the "queue", right? You think it is safe to rely on ZeroMQ to do this queuing thing? What would you recommend?Leo Cavalcante

2 Answers

0
votes

Aha! Thank you very much @Mjh for clarifying things.

Turns out that I do need something in the middle that should be a message queue broker and ZMQ does have something for that. Internally calling zmq_proxy starts a rrbroker loop, unfortunately theses guys aren't abstracted at reactphp/zmq, but they are at ext-php through ZMQDevice.

Source: http://zguide.zeromq.org/php:chapter2#ZeroMQ-s-Built-In-Proxy-Function

0
votes
(5555)                      (5556)
push -|  (5555) > (5556)   |- pull
push -|-> pull  &  push <- |- pull
push -|   bind  /  bind    |- pull
connect                    connect

This is very similar to ZMQPoll idea. Bad you do not required use reactphp/zmq

You can run process with ZMQPoll. This process can be bound on 5555 and 5556. And other process can push and pull messages.

my example how it can be work

more examples on official ZMQ site

PHP example poller from official site

UPD: If you want other solution, look at ZMQProxy