How to use WS-Discovery spec to discover ONVIF devices on a network in Java?

Question

I am trying to discover ONVIF devices with some Java code. Specifically, I am trying to get their device service address (which is just their IP address I believe?), as the ONVIF Core Spec notes (in Section 4.3) that "A successful discovery provides the device service address. Once a client has the device service address it can receive detailed device information through the device service...". Ultimately getting this detailed information of the ONVIF devices on the network is my goal. In general I'm also looking for some guidance relating to using the ONVIF Spec.

I am still new to the web service world (and networking in general), so forgive me if I say anything dumb. However, I have put in a lot of effort into this myself: I have read a good deal of ONVIF Core Spec, ONVIF Application Programmer's Guide, and WS-Discovery Specification. If I may I'll just summarize what I know so you can tell me if I'm on the right track:

1. "Web Services" is the name of a standard technology using platform and language independent web service standards such as XML, SOAP, and WSDL over an IP Network. The basic idea is that we want to be able to call what are effectively methods/functions (a service) from any programming language.
2. A web service is typically hosted on a server; but in the ONVIF use case, the web service provider is the ONVIF device (eg, an IP Camera). Thus to interact with the device from any language we use web service operations / calls, as web service calls can be implemented in any language.
3. XML is the data description syntax (used because it's language agnostic; any language can parse it). SOAP is the communication protocol used to get SOAP-infused XML documents back and forth (basically, make our method calls). WSDL is used for describing the services (it's an XML based description of the web services' interface). I have downloaded the WSDL for device management here, and generated via WSDL compiler wsimport (provided by JDK) the Java classes from the WSDL to use in my code. But I understand that calling these methods will come after device discovery, correct?
4. ONVIF Devices are discovered according WS-Discovery specification. You send a Probe message, and devices that match the Probe's constraints send back a ProbeMatch message, as described on page 13 and 14 in ONVIF Application Programmers Guide

Here is where I begin to get confused. How exactly do I send this message in Java? The ONVIF Application Programmer's Guide provides some pseudocode on page 15, but I can't figure out how to implement it. Section 4.3.1 in that guide specifically is what I'm stuck on. I understand that the "scopes" and "types" are just constraints you can embed in a probe, but they are not required (as per page 5 of WS discovery spec). Since I want to discover all devices I figure I need no constraints to start, right?

So that guide also provides a sample SOAP Message on page 110 used for discovery. Removing the type declaration out of it (because I don't want that constraint), I understand that my SOAP Message to send would be (I believe?) this:

<?xml version="1.0" encoding="UTF-8"?>
<e:Envelope xmlns:e="http://www.w3.org/2003/05/soap-envelope"
 xmlns:w="http://schemas.xmlsoap.org/ws/2004/08/addressing"
 xmlns:d="http://schemas.xmlsoap.org/ws/2005/04/discovery"
 xmlns:dn="http://www.onvif.org/ver10/network/wsdl">
 <e:Header>
 <w:MessageID>uuid:84ede3de-7dec-11d0-c360-f01234567890</w:MessageID>
 <w:To e:mustUnderstand="true">urn:schemas-xmlsoap-org:ws:2005:04:discovery</w:To>
 <w:Action
a:mustUnderstand="true">http://schemas.xmlsoap.org/ws/2005/04/discovery/Pr
obe</w:Action>
 </e:Header>
 <e:Body>
 <d:Probe>
 </d:Probe>
 </e:Body>
</e:Envelope>   

And I also understand the WS-Discovery technology used address 239.255.255.259 with UDP port 3702... but that's the end of what I get. How do I send that SOAP Message to that address and port in Java? How do I read the response (I think that I'll get back a ProbeMatch message, in the form of an SOAP-infused XML document, so I'll need to parse that XML to get the XAddrs, but not sure). Do I need to send a UDP broadcast of that SOAP message to that address somehow?

TL;DR: I believe to do ONVIF Device Discovery I need to send that SOAP message above to address 239.255.255.259 on UDP Port 3702. I have no clue how to do that in Java and was just looking for some guidance; I'm not even sure I'm on the right track to do device discovery.

Answer 1

Through much trial and error, I have found (another) solution to my problem. mpromonet's is probably the way to go in most cases, I just wanted to avoid using a sizable dependency like Apache. I also thought this should be doable with just some simple UDP messaging.

This solution is also based on SO user Thomas' helpful code here. I've mostly just simplified his code by removing the threading, and added some comments. Again his solution is probably better than mine (more performant); BUT, mine may be more understandable for a beginner (like me).

Here is the code:

import java.io.ByteArrayInputStream;
import java.io.File;
import java.io.IOException;
import java.net.*;

import javax.xml.namespace.QName;
import javax.xml.soap.*;
import java.util.*;

public class ONVIFDeviceDiscoveryFIN {

   // Following constants are related to Discovery process
   public static final int WS_DISCOVERY_TIMEOUT = 4000; // 4 seconds. Time to wait to receive a packet
   public static final int WS_DISCOVERY_PORT = 3702; 
   public static final String WS_DISCOVERY_ADDRESS_IPv4 = "239.255.255.250";

   // note that the probe below MUST be given a unique urn:uuid. Devices will NOT reply if the urn:uuid is not unique! 
   public static final String WS_DISCOVERY_PROBE_MESSAGE = "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\r\n" + 
        "<soap:Envelope xmlns:soap=\"http://www.w3.org/2003/05/soap-envelope\" xmlns:tds=\"http://www.onvif.org/ver10/device/wsdl\" xmlns:tns=\"http://schemas.xmlsoap.org/ws/2005/04/discovery\" xmlns:wsa=\"http://schemas.xmlsoap.org/ws/2004/08/addressing\">\r\n" + 
        "   <soap:Header>\r\n" + 
        "      <wsa:Action>http://schemas.xmlsoap.org/ws/2005/04/discovery/Probe</wsa:Action>\r\n" + 
        "      <wsa:MessageID>urn:uuid:5e1cec36-03b9-4d8b-9624-0c5283982a00</wsa:MessageID>\r\n" + 
        "      <wsa:To>urn:schemas-xmlsoap-org:ws:2005:04:discovery</wsa:To>\r\n" + 
        "   </soap:Header>\r\n" + 
        "   <soap:Body>\r\n" + 
        "      <tns:Probe>\r\n" + 
        "         <tns:Types>tds:Device</tns:Types>\r\n" + // Constraint to find just ONVIF devices hopefully? Recall we are sending a probe on the 192.168.0.50 network; if we have no constraints, it would find everything there! WS-Discovery generally is for much more than ONVIF, like printers and stuff
        "      </tns:Probe>\r\n" + 
        "   </soap:Body>\r\n" + 
        "</soap:Envelope>";

   private static ArrayList<String> getResponsesToProbe(String uuid) throws IOException{
       // TODO: add in ability to send scope and type constraints
       // NOTE: We do need to know the address of the network interface to discover devices on...
       // Function composes and sends a Probe to discover devices on the network. uuid is the urn:uuid to put in the probe. Functions returns all the SOAP-Infused XML responses (all the ProbeMatches).

       // give the probe a unique urn:uuid (we must do this for each probe!). This is generated outside function
       final String probe = WS_DISCOVERY_PROBE_MESSAGE.replaceAll("<wsa:MessageID>urn:uuid:.*</wsa:MessageID>", "<wsa:MessageID>urn:uuid:" + uuid + "</wsa:MessageID>");

       // set up the "sender and receiver"; this is the socket that we send our probe from, and where we receive back the ProbeMatch responses.
       // NOTE:  that we do need to know the address of the network interface to discover devices on... (port could be anything)
       final int port = 55000;
       DatagramSocket senderAndReceiver = new DatagramSocket(port, InetAddress.getByName("192.168.0.50")); // so you do need to know the address of your network interface to discover devices on...
       senderAndReceiver.setSoTimeout(WS_DISCOVERY_TIMEOUT);

       // send the probe 
       DatagramPacket probeMsg = new DatagramPacket(probe.getBytes(), probe.length(), InetAddress.getByName(WS_DISCOVERY_ADDRESS_IPv4), WS_DISCOVERY_PORT);
       senderAndReceiver.send(probeMsg);

       // read in the responses
       ArrayList<String> responses = new ArrayList(); // this is the collection of all SOAP-infused XML ProbeMatch responses
       byte[] receiverBuffer = new byte[8192];
       DatagramPacket receiverPacket = new DatagramPacket(receiverBuffer, receiverBuffer.length); // this is the packet that receive the response in. Get's updated with the next response on each call to .receive()
       while (true) {
           try {
               senderAndReceiver.receive(receiverPacket);
               responses.add(new String(receiverPacket.getData()));

           } catch (SocketTimeoutException e) {
               // System.out.println("Socket read timeout; taken to mean that there is no more responses -- i.e., no more Probe Matches");
               break;
           }
       }

       // close the socket
       senderAndReceiver.close();

       return responses;

   }

   public static void main(String[] args) throws IOException, SOAPException {


       final String uuid = UUID.randomUUID().toString(); // generate the uuid to add to the Probe message

       ArrayList<String> responses = getResponsesToProbe(uuid); // responses is a collection of all the SOAP-infused XML ProbeMatches . It's all of our responses to the probe; it's basically the devices we've discovered!

    }

}

Some notes on using this:

1. To use this solution, you need to know the "network interface" to look on. In my code, this is the 192.168.0.50 . This is the network that my camera that I'm looking to discover is on. To find this, run an arp -a command in your cmd prompt (Not sure how to do this on Mac or Linux), and locate the IP of your camera. The interface that it falls under is the one you'll want to use as that "192.168.0.50". In my limited understanding, these interfaces basically segment your network, so you need to pick the right one to look for devices on. I think (?) Thomas' code avoids this problem by finding all these network interfaces.This is done in lines 81-100 in his code.

2. You MUST give your Probe a unique UUID when you send it. This was one of my errors in doing this; I tested with a hardcoded UUID in the Probe (in the WS_DISCOVERY_PROBE_MESSAGE). This will work to discover devices ONCE; but after that, if you send a probe with the same UUID, it seems the devices won't reply at all. You'll get no error response either, hence why this was tough fro me to find out. It's as if the device keeps some internal log of the UUIDs of all the Probe's its received; and if you send a probe with an old UUID, it just rejects it. Or at least, this is the case for the ONVIF-compliant camera I'm testing with (an AXIS M3045-V). I'm not sure if this behavior is required by the ONVIF spec, but it is at least apparent in an AXIS M3045-V.

3. Notice: SOAP normally relies on HTTP for transfer; but here we use it on top of UDP.

I hope this helps anyone trying to do something similar. Let me know if there's anything I can do to help; I've read tons of documentation at this point, so I may be able to lend a hand!

Answer 2

Using CXF WSDiscoveryClient you can probe ONVIF Device.
By default WSDiscoveryClient use WS-discovery 1.1 and ONVIF use WS-discovery 1.0, so you need to enable WS-discovery 1.0.
A short implementation could be :

import java.util.List;
import javax.xml.ws.EndpointReference;
import org.apache.cxf.ws.discovery.WSDiscoveryClient;

public class Main 
{
    public static void main(String[] args) 
    {
        WSDiscoveryClient client = new WSDiscoveryClient();
        client.setVersion10(); // use WS-discovery 1.0
        client.setDefaultProbeTimeout(1000); // timeout 1s

        System.out.println("Probe:" + client.getAddress());
        List<EndpointReference> references = client.probe();

        System.out.println("Nb answsers:" + references.size());
        for (EndpointReference ref : references)
        {
            System.out.println(ref.toString());
        }
    }
}