I guess the reason is the second IP fragment are not with UDP header(I think it is the same for TCP), so libpcap can't capture them using the filter express udp port 20000.
Yes, that is correct.
You might try udp port 20000 or (ip[6:2] & 0x1fff) != 0
, which will capture packets to or from port 20000 and IP fragments other than the first fragment; that's not ideal, but it's all you can do with libpcap filters, given that the filter mechanism it uses (which is part of the OS kernel) does not maintain any history between packets, and therefore has no way of knowing that a packet with a given IP ID happens to be part of the same fragment as another packet with the same IP ID, a fragment offset of 0, and a UDP header with port 20000.
(Note also that at least some versions of Linux would transmit fragments of an IP datagram in reverse order - in order to allow the recipient to see the last fragment first, and thus to be able to more often correctly estimate the size of the reassembled packet. That would make it even more difficult to capture all the fragments of IP packets with a filter that checks fields in the TCP or UDP header.)