ZeroMQ and actor model

2
votes

I'm having problems scaling up an application that uses the actor model and zeromq. To put it simply: I'm trying to create thousands of threads that communicate via sockets. Similar to what one would do with a Erlang-type message passing. I'm not doing it for multicore/performance reasons, but because framing it in this way gives me very clean code.

From a philosophical point of view it sounds as if this is what zmq developers would like to achieve, e.g. http://zeromq.org/whitepapers:multithreading-magic

However, it seems as if there are some practical limitations. At 1024 inproc sockets I start getting the "ZMQError: Too many open files" error. TCP gives me the typical "Assertion failed: fds.size () <= FD_SETSIZE" crash.

Why does inproc sockets have this limit? To get it to work I've had to group together items to share a socket. Is there a better way?

Is zmq just the wrong tool for this kind of job? i.e. it's still more a network library than an actor message passing library?

1
zeromqpyzmqakka-zeromq
Try increasing the number of fds limut via ulimit – Has QUIT--Anony-Mousse
inproc limit comes from OS/kernel sizing of this resource descriptor table. ZeroMQ is a great framework, so worth in actor-based systems. You will need to dig deeper in kernel adjustments and ZMQ-IO-thread internals once tuning gross-scales/latencies/performance – user3666197
Unfortunately, it has to run on windows :/ Hopefully I can change the limit. I really like ZeroMQ, especially the way it's allowing me to separate complex application behavior into components that don't share state, so these limits are somewhat annoying. – janto

1 Answers

0
votes

ZMQ uses file descriptors as the "resource unit" for inproc connections. There is a limit for file descriptors set by the OS, you should be able to modify that (found several potential avenues for Windows with a quick Google search), though I don't know what the performance impact might be.

It looks like this is related to the ZMQ library using C code that is portable among systems for opening new files, rather than Windows native code that doesn't suffer from this same limitation.