ZMQ C++ Req to Router issues

Question

I am using ZeroMQ for my network layer and so far everything works except when it comes to ROUTER sockets. In particular I receive the expected message on the ROUTER but when I try send an answer back to my REQ socket the message is never received.

Here is a relatively simple test I wrote which tries to send a "HELLO" message to the ROUTER and expects a message to come back.

here the client code :

    try
    {
        zmq::context_t myContext;

        zmq::socket_t reqSocket(myContext, ZMQ_REQ);
        reqSocket.setsockopt(ZMQ_IDENTITY, "REQ", 3);
        reqSocket.connect(gpRouterAddress);

        //request delimiter
        zmq::message_t zmqMsgReqDelimiter(1);
        memcpy ((void *) zmqMsgReqDelimiter.data(), "\0", 1);
        reqSocket.send(zmqMsgReqDelimiter, ZMQ_SNDMORE);

        //some message
        zmq::message_t reqMsg(5);
        memcpy ((void *) reqMsg.data(), "HELLO", 5);
        reqSocket.send(reqMsg, 0);

        int rcvMore = 0;
        size_t sizeInt = sizeof(int);
        bool bRcvMore = true;
        std::vector<std::string> history;

        while(bRcvMore)
        {
            zmq::message_t zmqMsg;

            reqSocket.recv(&zmqMsg, rcvMore);
            const char* pMsgStr = static_cast<char*>(zmqMsg.data());
            history.push_back(pMsgStr);

            reqSocket.getsockopt(ZMQ_RCVMORE, &rcvMore, &sizeInt);

            bRcvMore = (rcvMore == 1);
        }
    }
    catch (zmq::error_t error)
    {
        std::string errorStr = error.what();
    }

and here is my Router code (can run in a different thread , in which case theContext would be the same as "myContext" from the code above) or a entirely different application :

    try
    {
        zmq::context_t theContext;

        zmq::socket_t router (theContext, ZMQ_ROUTER);
        int value = 1;
        router.setsockopt(ZMQ_ROUTER_MANDATORY, &value, sizeof(int));
        router.setsockopt(ZMQ_IDENTITY, "ROUT", 4);
        router.bind(gpRouterAddress);

        zmq::message_t zmqMsgInternalAddress;
        router.recv(&zmqMsgInternalAddress, 0);
        const char* pAddressStr = static_cast<char*>(zmqMsgInternalAddress.data());

        zmq::message_t zmqMsgDelimiter;
        router.recv(&zmqMsgDelimiter, ZMQ_RCVMORE);
        const char* pDelimiterStr = static_cast<char*>(zmqMsgDelimiter.data());

        int rcvMore = 0;
        size_t sizeInt = sizeof(int);
        bool bRcvMore = true;

        router.getsockopt(ZMQ_RCVMORE, &rcvMore, &sizeInt);
        bRcvMore = (rcvMore == 1);

        std::vector<std::string> history;

        while(bRcvMore)
        {
            zmq::message_t zmqMsg;

            router.recv(&zmqMsg, rcvMore);
            const char* pMsgStr = static_cast<char*>(zmqMsg.data());
            history.push_back(pMsgStr);

            router.getsockopt(ZMQ_RCVMORE, &rcvMore, &sizeInt);

            bRcvMore = (rcvMore == 1);
        }


        //reply address

        size_t len = strlen(pAddressStr) - 1; //if I don't subtract 1 char here, an exception will be raised 

        zmq::message_t replyAddress(len);
        memcpy ((void *) replyAddress.data(), pAddressStr, len);
        router.send(replyAddress, ZMQ_SNDMORE);

        //reply delimiter
        zmq::message_t zmqMsgReplyDelimiter(1);
        memcpy ((void *) zmqMsgReplyDelimiter.data(), "\0", 1);
        router.send(zmqMsgReplyDelimiter, ZMQ_SNDMORE);

        //some message
        zmq::message_t replyMsg(5);
        memcpy ((void *) replyMsg.data(), "WORLD", 5);
        router.send(replyMsg, 0);

    }
    catch (zmq::error_t error)
    {
        std::string errorStr = error.what();
    }

I receive the "HELLO" message on the router and I can step through the ROUTER send and everything seems to be sent okay (i.e. no exception is being raised) but I never receive a message on the REQ socket which will keep waiting indefinitely.

According to the ZeroMQ Guide I should expect the ROUTER to receive the following messages :

The REQ socket sends

empty
HELLO

the ROUTER receives

REQ
empty
HELLO

but I receive

REQ
some binary message
empty
HELLO

and I send

REQ
empty
WORLD

which I would expect to arrive at REQ as

empty
WORLD

If I connect to a REP socket instead (using a simple REQ-REP topology everything works fine).

Can anyone see what I am missing/doing wrong?

This could be useful to others making the same mistake, but you should probably shorten the question now that the example code isn't needed and be more specific to what the actual problem was. — JSON

user3228912 user3228912 · Accepted Answer · 2014-01-24T22:16:12

I found the issue.

Basically the error was in how I sent the delimiter

    zmq::message_t zmqMsgReplyDelimiter(1);
    memcpy ((void *) zmqMsgReplyDelimiter.data(), "\0", 1);
    router.send(zmqMsgReplyDelimiter, ZMQ_SNDMORE);

it should just be

    zmq::message_t zmqMsgReplyDelimiter(0);
    router.send(zmqMsgReplyDelimiter, ZMQ_SNDMORE);

ZMQ C++ Req to Router issues

1 Answers