Linux: Raw Sockets Sent Packets Not Received Locally Under KVM

Question

I've been trying to send UDP packets using a raw socket, however, the sent packets are not received locally. The same packets are received if sent to a remote destination. The test is performed under KVM. The same test seems to be working under Parallels.

The socket is set up with:

raw_socket = socket (AF_PACKET, SOCK_RAW, htons (ETH_P_ALL));

There's a receive filter attached and it is set as promiscuous. Packets are received correctly. Sent packets ARE received in Wireshark and analysed correctly. However, the local program (e.g. nc -l -u -p ...) does not receive the packet. If I send the same packet to a remote destination, the packet is received correctly. The fact that Wireshark receives the packet suggests that it is not dropped by any iptables rules (I've also checked all DROP rules counters and there are no dropped packets).

It seems as if the packet is routed but not reprocessed by the network stack, if that even makes sense.

It's supposed to be an example code in a network course, and currently, I'm losing face in front of my students. Hope none of them are on this list ;-).

Would appreciate any help, Yuval.

A few comments about the code. For brevity, the UDP checksum is set to 0, the IP checksum is calculated from the header. Both are validated by Wireshark (and accepted by the remote machine on KVM as well). The code parses a message and sends a reply on the buffer it received. udp_hdr and ip_hdr point to the corresponding headers in the packet buffer (buffer), while payload points to the start of the L7 data.

Here's the code: { /* Process the request. */ ssize_t msg_len; int max_len = sizeof (buffer) - (payload - buffer); int tmp_port; in_addr_t tmp_ip; struct ether_addr tmp_ether;

        msg_len = parse_and_send (ctx, payload, size, max_len);
        if (msg_len < 0)
        {
           /* Error */
           error_print (ctx, "Failed to parse and construct answer.");
       continue;
        }
        if (msg_len == 0)
        {
           /* No message to send. */
           continue;
        }

        /* Switch sources & destinations and update payload lengths. */
        tmp_port = udp_hdr->dest;
        udp_hdr->dest = udp_hdr->source;
        udp_hdr->source = tmp_port;
        udp_hdr->len = htons (msg_len +
                              (payload - (const uint8_t*) udp_hdr));
        tmp_ip = ip_hdr->daddr;
        compute_udp_cksum (udp_hdr);

        ip_hdr->daddr = ip_hdr->saddr;
    ip_hdr->saddr = tmp_ip;
        ip_hdr->tot_len = htons (msg_len +
                                 (payload - (const uint8_t*) ip_hdr));
        compute_ip_cksum (ip_hdr);

        memcpy (tmp_ether.ether_addr_octet, ether_hdr->ether_dhost,
                sizeof (tmp_ether));
        memcpy (ether_hdr->ether_dhost, ether_hdr->ether_shost, ETH_ALEN);
        memcpy (ether_hdr->ether_shost, tmp_ether.ether_addr_octet,
                sizeof (tmp_ether));

        if (sendto (ctx->raw_socket,
                    &buffer, msg_len + (payload - buffer), 0,
                    (struct sockaddr*) &ll_addr, sizeof (ll_addr)) < 0)
        {
           ERROR_SYSTEM (ctx, "Sending packet");
           error_print (ctx, "Failed to send packet");
       continue;
        }
     }

If the packets are received by Wireshark it does not mean that they are not dropped; but if you checked it it is fine. Please mention exactly the command that you are using for receiving packets in the shell. — gst
To make it easiest, I used nc -l -u -p 2222. On a remote machine it works fine. On the local machine (with the raw socket), nothing is received. — Yuval
Can you give the code that you are using for sending packets? — gst
Apparently here's the code above. Can't get the code into a comment... Thanks. — Yuval

gst gst · Accepted Answer · 2014-05-23T10:44:57

Lets write down the facts that we know:

1- You told that you can see the packets in the packet capture programe (e.g Wireshark)

2- You told that you can not receive the packets using netcat when you send it to the loopback interface.

3- You told that you can receive the packets when you send them to a remote machine.

So the problem is either in your code or your local machine. I have a bunch of suggestions:

1- Test the code on another machine, and send the packets to the loopback interface and try to capture them using netcat.

or

2- Try to send a bunch of UDP packets using netcat to the local interface and receive them using another netcat instance and check if it works; if it worked you know that the problem is in your code otherwise the problem can be because of your local machine.

Linux: Raw Sockets Sent Packets Not Received Locally Under KVM

1 Answers