Send messages to ZeroMQ server using conventional TCP, possible?

Question

I'm not sure if I'm doing this right, but I would like to be able to send messages to my server running ZMQ from normal TCP connections. The server is running Python ZMQ on port 5555 using a TCP transport. I would like to be able to send messages to it using different clients (Python, Java, PHP) that use conventional TCP. This is what I have so far:

SERVER

context = zmq.Context()
socket = context.socket(zmq.REP)
socket.bind("tcp://*:5555")

while True:
    message = socket.recv()
    print message
    socket.send('{"name":"someone"}')

CLIENT

s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect(('127.0.0.1', 5555))
s.send('Hello, World!')
data = s.recv(1024)
print data

Printing data on the client does not give me the message I am expecting. I get this: �. I tried doing bytes(data).decode('utf8') thinking what I'm getting was an array of bytes, but I get the following error:

UnicodeDecodeError: 'ascii' codec can't decode byte 0xff in position 0: ordinal not in range(128)

I am just wondering: Is this possible at all? Or am I doing something wrong? Also, is it recommended?

My reason for not using ZMQ on the clients is that I want to reduce the number of dependencies (ZeroMQ being one).

Thank you for your help.

abarnert abarnert · Accepted Answer · 2014-10-07T23:43:56

ZeroMQ is a protocol that sits on top of TCP (well, technically, ZMTP is a protocol; 0MQ is the library that implements the ZMTP protocol, and ZeroMQ is the Python implementation of 0MQ…). All the signaling it does, and all the framing (dividing the stream of bytes into separate messages), is done by sending bytes over the socket. A client that doesn't know how to do ZeroMQ signaling and framing is just going to see a bunch of garbage.

What you're trying to do is exactly like trying to write a web client that just reads off a socket without knowing anything about HTTP. You're going to get a bunch of framing stuff that you don't know what to do with. The only difference is that in the case of HTTP, the framing is sometimes (but not always—you can have MIME envelopes, gzip transport-encoding, chunked transport, …) just a bunch of human-readable ASCII that comes before the data, while with ZMTP it's never human-readable…

If you want to send data over a plain TCP socket, you have to do that by creating a plain TCP socket on the server side and calling its send (or sendall, etc.) method. If you want to send data over a ZMTP channel, you have to do that by parsing ZMTP, or using a library that does it for you (like ZeroMQ), on the client side.

One more thing to keep in mind is that, unlike ZMTP, TCP is not a message-oriented protocol; all TCP sends is a stream of bytes. From the receiving side, there's no way to know when one send ends and the next one begins. So, for almost anything but a "send a request, get a response, hang up" protocol, you need to write some kind of framing of your own. This can be as simple as "messages are strings that have no newlines in them, and each message is separated by a newline" (in which case you can just use socket.makefile), but often the message format has to be more complicated, or you have to send "commands" rather than just data, etc.

Since tdelaney's answer has been deleted (which I think means it's invisible to anyone under 10K rep), and had a useful suggestion, I'll repeat it here, with credit as due: You can (using the ZeroMQ library) write a piece of middleware that talks ZMTP to the server but your own simple TCP-based protocol to the clients. ZeroMQ has been specifically designed to make this reasonably easy; as tdelaney put it, it's "kinda like Lego, you build a robust communication infrastructure by building different communicating parts. Not all of the parts need to be zeromq."

Send messages to ZeroMQ server using conventional TCP, possible?

1 Answers