I don't think the previously accepted answer necessarily does what you think it does and possibly not even what you want it to do. The original question stated, "But it results in very big file within minutes, Can i filter tcpdump on base of topic name"
If you're trying to limit the size of the capture file, then the previously accepted answer isn't doing that because it uses the exact same capture filter as was originally provided, namely src 10.x.x.x. This means that you're capturing the same amount of data as you were before. Just because a capture file name wasn't specified doesn't mean that packets aren't being written to a file; they are. In the case of tshark, packets are written to a temporary file, which will continue to grow until the capture session is terminated and then ideally it will be deleted, but not always. The location of the temporary file varies depending on the platform that tshark is run on, but you should be able to easily locate the directory by running tshark -G folders | grep "^Temp".
Now, if you want to reduce the size of the capture file, or the number of packets that you see, then you should be able to modify the tcpdump or tshark command-line arguments to accomplish that.
First off, if you don't need the entire payload, you can apply a snaplen to cut the packets short after some appropriate value. This is done using the -s option, and it's the same option for either capture tool.
OK, but whether you decide to apply a snaplen or not, if you want to filter based on the specific topic name, most likely you can achieve this; however there are a couple of caveats that I listed below. The main idea is to use the slice operator, [] (see the pcap-filter man page) to compare various bytes of the TCP payload to specific values. (NOTE: Neither tcpdump itself nor pcap-filter refers to this operator as the slice operator, but wireshark-filter does, so I do as well.) So the filter should:
- Match packets only to/from a particular host, in this case 10.x.x.x
- Match only MQTT packets (typically by port number, which I'll assume to be the standard tcp/1883 port)
- Match only PUBLISH messages with QoS 0
- Match only PUBLISH messages where the topic length is 26 bytes
- Match only PUBLISH messages where the topic is "PKGCTRL/1/status/frequency"
Here's a command that should work (at least in most cases -> see caveats below):
tcpdump -i team0 -w mqtt-trace.pcap \
"(src host 10.x.x.x) and \
(tcp port 1883) and \
((tcp[20]&0xf6)=0x30) and \
(tcp[22:2]=26) and \
(tcp[24:4]=0x504b4743 and tcp[28:4]=0x54524c2f and \
tcp[32:4]=0x312f7374 and tcp[36:4]=0x61747573 and \
tcp[40:4]=0x2f667265 and tcp[44:4]=0x7175656e and tcp[48:2]=0x6379)"
It should be obvious from the description of the desired filter above what each separate component of the filter is doing for you.
Here, you'll end up with a capture file of the desired packets that you can reference later on if you need to. You can even apply this same filter to the tshark solution too if you prefer as well as writing to the named capture file, because as I explained earlier, tshark is writing packets to a file whether you explicitly specify one or not.
Caveats:
The filter assumes TCP headers are 20 bytes for simplicity in illustrating the solution, but that may not be the case. If you want a more robust solution to accommodate any TCP header size, then you'll need to determine the TCP header size from the data offset field of the TCP header, which is done in the filter using ((tcp[12]&0xf0)>>4)*4, and then replace every occurrence of 20 in the slice operator's offset field with that value. So for example, tcp[22:2]=26 becomes tcp[(((tcp[12]&0xf0)>>4)*4)+2:2]=26, etc.
Because the MQTT remaining message length field is variable-length encoded per MQTT3.1.1 section 2.2.3 Remaining Length, the filter, as provided above, will only work for values of the remaining length field from 0 to 127, i.e., the remaining length field must only be a single byte. Given that the topic in this case is 26 bytes and the topic length itself is 2 bytes, this means the filter will only work for MQTT message payloads of 99 bytes or less (127 - (2 + 26)).
