I am working on a network management application that pings managed hosts using ICMP via raw BSD sockets (sendto).
My question is not how to do that, in fact it works very well. I only seem to hit on a problem when the target host is sending Destination unreachable messages to the management server while I it is trying to send an ICMP echo request to that host.
When a host is added in the management application, it will be tested using said ICMP ping and at the same time, a "discovery" is started, testing various ports on the host such as SNMP (161) etc., using the respective protocols required to do so.
Now, I am aware that it does not make much sense to perform that "discovery" on a host where I do not know whether it exists or responds (ie it sent an ICMP echo reply), but for this question this is crucial. Let us also assume that the hosts in question here are actually reachable and do respond to ICMP pings, e.g. via the command line ping.
The "discovery" eventually causes some ICMP "Destination unreachable: Port unreachable" packets to be sent back to the management server, e.g. when there is no SNMP agent running on the target host and port 161 is, in fact, not reachable.
This is perfectly fine, but for some reason, this seems to hinder the ICMP socket from sending out the ICMP echo request to that host, which is required for the ICMP ping. Since that request is never sent, my application gets no reply, and the host will be deemed "unreachable" after a while (timeout).
I analyzed the network traffic using Wireshark, thus I can tell there is actually no ICMP echo request sent to the target host. The application I work on also has a "status poll" feature which can be used to "manually" perform another ICMP ping on that host. If this is used after the host has been added (ie no more Destination unreachable packets are incoming), the echo request is sent out without any problems.
I am puzzled about the ICMP echo request not being sent in the situation I explained above. I do have another ICMP socket that receives all incoming ICMP packets (in order to react to replies), but it should not affect the sending socket, should it? In my code, there is no reaction defined for Destination unreachable messages, they are simply ignored, so I am very certain that it is not my own code that disables the request sending.
In fact, the sendto function that is responsible for sending out the request is executed (made sure of that by debugging) and even returns a success (number of bytes sent), but the packet is not actually being sent. I can reproduce this on both Windows and Linux systems, so I don't believe it's any kind of an operating system problem.
Disabling the "discovery" that causes those Destination unreachable messages to be sent back makes everything work fine again, so I am fairly certain that it's those messages that hinder the ICMP echo request from being sent. The big question I fail to find an answer for is: Why is that?
I can provide more information if needed, however, I'm afraid that I am not allowed to post any code here, because this is a commercial product. The code is nothing special, though, just sending and receiving ICMP (IPv4) packets with TOS=0 and TTL=64 using raw ICMP sockets.
