Wireshark error when opening .pcap dumped with pcapy

Question

I'm using the following piece of python code to capture traffic and dump it to a .pcap file:

from pcapy import open_live

p = open_live("eth0", 65535, 1, 0)
dumper = p.dump_open("./test.pcap")
while capturing:
    (header, packet) = p.next()
    dumper.dump(header, packet)

I'm actually running this in a thread, where capturing is a threading.Event() that is set to False when I want to stop the capture (so it exits the loop and returns cleanly).

However, when I try to open the test.pcap with wireshark, I get this message:

The capture file appears to have been cut short in the middle of a packet.

I sometimes see that there are a couple of packets missing at the end of the test.pcap (I can debug that because I'm writing the intercepted packets in a .csv file). But, besides that, I think the pcap file is fine. This message is a bit annoying, though. I thought that it could be that I need to include some magic number to make wireshark believe it's a wireshark capture or something like that. I found other questions of people getting this message because they don't close the capture cleanly (but as I said, I do (or do I need to explicitly close the file descriptor open by pcapy? I couldn't find a method in the pcapy API that closes it). Also, I'm not capturing with wireshark, so it might be a different problem).

Does anybody know what is the message due to? Or, does anybody know how can I debug and find the cause that makes wireshark pop this message?

EDIT

Pcapy source code that closes dumper here.

Answer 1

pcapy tries to mimic the API of the pcap library. Unfortunately it is missing an important function: pcap_dump_flush. By default pcap buffers writes and will only write the buffer to disk if it it contains enough data or if the dump file gets closed. With pcap_dump_flush on can trigger the flushing of the buffer. Since this function is not available from pcapy your only choice is to close the file before reading it from another application.

Answer 2

I found other questions of people getting this message because they don't close the capture cleanly

What's not being closed cleanly is the handle for the dumper.

or do I need to explicitly close the file descriptor open by pcapy?

You need to close dumper.

I couldn't find a method in the pcapy API that closes it

There isn't one. From a quick look at the pcapy source code, it's closed implicitly when dumper gets released. That should be happening if your Python program exits cleanly.

If it's not exiting cleanly (for example, if it's being killed by control-C without it, or the Python interpreter, catching the signal and cleanly exiting), you need to fix that.

If it is exiting cleanly, there's probably a bug somewhere in pcapy; report that as a bug to the pcapy developers.

(pcap_dumper_ts use the "standard I/O library" C routines, which means they do buffered output, and data doesn't get written out to the file immediately with every dump call; closing the dumper causes data not yet written to the file to be written out. That should be done when the C program - which would be the Python interpreter - exits, if it's not done before that.)