I want to propose or atleast get a discussion started about adding device discovery specification to haystack.

I believe using a ZeroConf mechanism would do well with haystack enabled products.

This would include supporting 2 services.

1. mDNS - Multicast DNS
2. DNS-SD - DNS Service Discovery

Multicast DNS will be the discovery mechanism -- I believe we should do a Continuous Query with Known-Answer Suppression mechanism. We could also provide a feature where after querying stops, it could listen for service advertisements and detect new haystack enabled devices that are added to the network.

This would also add security because enables devices to be accessed via host name such as haystackserver.local, centralplant1.local, and etc; this would enable devices to have a verified SSL certificate instead of only using a self-signed one. device-device and pc-device communication can work with names instead of IP addresses.

This also provides immunity to a dynamic IP changing via DHCP.

DNS Service Discovery allows the devices to advertise what services they provide. This could provide the host and port via a SRV record and Haystack Version and URL to "about" via a key/value paired TXT record. A PTR record can be included for reverse DNS.

I think discovery would be a nice addition.

My initial impression is that using DNS with a PTR record might be too low level. I don't think most devices get mapped into DNS, so seems like it would be a lot of friction to use it in typical deployments

Do you have other suggestions that might be higher level? I am not familiar with what other standards are out there

With mDNS (Multicast DNS), the device acts as its own DNS server. All devices will respond to queries to a multicast group. They will respond with their DNS information which include the PTR, TXT, and a few records that mDNS defines.

This enables DNS ability without having a DNS server on the network. and combined with DNS-SD (DNS Service Discovery) we can discover the services such as haystack in the device.

I was thinking we would go with a ZeroConf standard which is this. Another option is UPNP.

I think mDNS has some advantages over UPNP as we could also begin to use devices by name and secure the networks with valid certificates. Devices can even be secured with wildcard certs where they answer up as device1.skyfoundry.local, device2.skyfoundry.local, and etc...

So with this, there isn't anything the SI has to do. The Haystack Server that supports discovery will listen for mDNS queries in the multicast group and respond. The consumer will issue mDNS queries to discover Haystack enabled devices.

This should be completely transparent to the SI and as if there is simply a custom discovery mechanism.

I think some of the other ZeroConf options are very difficult such as IPv4LL or proprietary such as WiSe-Zeroconf and Apple's Bonjour. I believe Bonjour has evolved to mDNS+DNS-SD.

Just to sum it all up. mDNS enables each device to respond to its own DNS queries. a query for all is used to discover all devices. DNS-SD enables the device to advertise its services. This is to be implemented by the Haystack Server and require Zero Configuration from the SI's side.

I agree with Michael, mDNS is fairly established and easy to use. Way better than UpNP (no xml and soap to start with :) Also initiatives like web-of-thing discovery use it (and others).

This may be fine for most private networks or small systems that only have these devices connected, but it we don't allow either one of these on our network here (large educational institution).

@Doug That is interesting. In a case like yours, is there any discovery mechanism that will work? I'm assuming that multicast packets are completely blocked and probably broadcast as well; or does your rules particularly target and block mDNS.

Do you see these institutions enabling mDNS in the future if it increases security? for example, enables PKI authentication with valid certificates?

Michael,

I'd like to throw my support behind this idea and conversation. I actually spent a few nights during COVID lockdowns looking into this and learning about mDNS and DNS-SD. I played with setting these things on my Mac locally and using some off the shelf apps to validate my configurations.

Notwithstanding the other comments, and I do understand the challenges of getting something like this implemented on larger networks, I strongly support the idea of this sort of feature/function becoming part of the Project Haystack technology landscape that can be used to enable automatic discovery of devices and services.

I look forward to seeing this develop and I am happy to assist if I can.

regards, Richard

@Michael I'll check with one of the network architecture people (I'm in the facilities department, so all of my network knowledge comes in second hand). I know that some of our lighting systems do not work well because they rely on multicast and we are still working on some issues with those in a few buildings.