Capturing with pcap can't keep up?

Question

For a small tool that I am building for OSX, I want to capture the lengths of packets send and received from a certain ethernet controller.

When I fetch the ethernet cards I also get extra information like maximum packet sizes, link speeds etc.

When I start the (what I call) 'trafficMonitor' I launch it like this:

static void initializeTrafficMonitor(const char* interfaceName, int packetSize) {
    char errbuf[PCAP_ERRBUF_SIZE];
    pcap_t* sessionHandle = pcap_open_live(interfaceName, packetSize, 1, 100, errbuf);
    if (sessionHandle == NULL)
    {
        printf("Error opening session for device %s: %s\n", interfaceName, errbuf);
        return;
    }

    pcap_loop(sessionHandle, -1, packetReceived, NULL);
}

The supplied interfaceName is the BSD name of the interface, for example en0. The packetSize variable is an integer where I supply the maximum packetsize for that ethernet adapter (that seemed logical at that time). For example the packetsize for my WiFi adapter is 1538.

My callback method is called packetReceived and looks like this:

void packetReceived(u_char* args, const struct pcap_pkthdr* header, const u_char* packet) {

    struct pcap_work_item* item = malloc(sizeof(struct pcap_pkthdr) + header->caplen);
    item->header = *header;
    memcpy(item->data, packet, header->caplen);
    threadpool_add(threadPool, handlePacket, item, 0);
}

I stuff all the properties for my packet in a new struct and launch a worker thread to analyze the packet and process the results. This is to not keep pcap waiting and is an attempt to fix this problem which already existed before adding this worker thread method.

The handlePacket method is like this:

    void handlePacket(void* args) { 

        const struct pcap_work_item* workItem = args;
        const struct sniff_ethernet* ethernet = (struct sniff_ethernet*)(workItem->data);

        u_int size_ip;
        const struct sniff_ip* ip = (struct sniff_ip*)(workItem->data + SIZE_ETHERNET);
        size_ip = IP_HL(ip) * 4;
        if (size_ip < 20) {
            return;
        }
        const u_int16_t type = ether_packet(&workItem->header, workItem->data);
        switch (ntohs(type)) {
            case ETHERTYPE_IP: {

                char sourceIP[INET_ADDRSTRLEN];
                char destIP[INET_ADDRSTRLEN];

                inet_ntop(AF_INET, &ip->ip_src, sourceIP, sizeof(sourceIP));
                inet_ntop(AF_INET, &ip->ip_dst, destIP, sizeof(destIP));

                [refToSelf registerPacketTransferFromSource:sourceIP destinationIP:destIP packetLength:workItem->header.caplen packetType:ethernet->ether_type];

                break;
            }
            case ETHERTYPE_IPV6: {
                // handle v6
                char sourceIP[INET6_ADDRSTRLEN];
                char destIP[INET6_ADDRSTRLEN];

                inet_ntop(AF_INET6, &ip->ip_src, sourceIP, sizeof(sourceIP));
                inet_ntop(AF_INET6, &ip->ip_dst, destIP, sizeof(destIP));

                [refToSelf registerPacketTransferFromSource:sourceIP destinationIP:destIP packetLength:workItem->header.caplen packetType:ethernet->ether_type];

                break;
            }
        }
}

Based on the type of ethernet packet I try to figure out if it is an packet send using an IPv4 or IPv6 address. After that is determined I send some details to an objectiveC method (Source IP address, Destination IP address and packet length).

I cast the packet to the structs explained on the website of tcpdump (http://www.tcpdump.org/pcap.html).

The problem is that pcap either does not seem to keep up with the packets received/send. Either I am not sniffing all the packets or the packet lengths are wrong.

Does anyone have any pointers where I need to adjust my code to make pcap catch them all or where I have some sort of problem.

These methods are called from my objectiveC application and the refToSelf is a reference to a objC class.

Edit: I am calling the initializeTrafficMonitor in a background thread, because the pcap_loop is blocking.

Unknown Unknown · Accepted Answer · 2012-09-02T23:29:28

On which version of OS X is this? In releases prior to Lion, the default buffer size for libpcap on systems using BPF, such as OS X, was 32K bytes; 1992 called, they want their 4MB workstations and 10Mb Ethernets back. In Lion, Apple updated libpcap to version 1.1.1; in libpcap 1.1.0, the default BPF buffer size was increased to 512MB (the maximum value in most if not all systems that have BPF).

If this is Snow Leopard, try switching to the new pcap_create()/pcap_activate() API, and use pcap_set_buffer_size() to set the buffer size to 512MB. If this is Lion or later, that won't make a difference.

That won't help if your program can't keep up with the average packet rate, but it will, at least, mean fewer packet drops if there are temporary bursts that exceed the average.

If your program can't keep up with the average packet rate, then, if you only want the IP addresses of the packets, try setting the snapshot length (which you call "packetSize"`) to a value large enough to capture only the Ethernet header and the IP addresses for IPv4 and IPv6. For IPv4, 34 bytes would be sufficient (libpcap or BPF might round that up to a larger value), as that's 14 bytes of Ethernet header + 20 bytes of IPv4 header without options. For IPv6, it's 54 bytes, for 14 bytes of Ethernet header and 40 bytes of IPv6 header. So use a packetSize value of 54.

Note that, in this case, you should use the len field, NOT the caplen field, of the struct pcap_pkthdr, to calculate the packet length. caplen is the amount of data that was captured, and will be no larger than the specified snapshot length; len is the length "on the wire".

Also, you might want to try running pcap_loop() and all the processing in the same thread, and avoid allocating a buffer for the packet data and copying it, to see if that speeds the processing up. If you have to do them in separate threads, make sure you free the packet data when you're done with it.

Capturing with pcap can't keep up?

1 Answers