Use of AWS IOT for indirectly connected devices

4
votes

I am thinking about using AWS IoT for an application, where there are several thousands of small bitmap displays (connected with proprietary wireless protocol) behind (possibly hundreds of) distributed gateways (PCs or Raspberry Pi's).

So far I have come up with the following concept:

  1. The gateway PC terminates the MQTT session. It has no own device table.

  2. thingname = <gatewayId>_<displayId>

  3. Display bitmaps are stored on S3 (filename = thingname)

  4. Updating the displays is just replacing the S3 file and then updating the bitmap version (eg SHA) in the desired state of the device shadow.

  5. The gateway would have to subscribe to the updates like /update/<gatewayId>/#

  6. There will be a rule that republishes from /update/<gatewayId>_<displayId> to /update/<gatewayId>/<displayId> (because thingnames cannot contain slashes and wildcards in MQTT must be complete path components)

  7. For each received message, the gateway will download the bitmap from S3, send it to the display and then update the reported state to the new version.

How do I handle disconnected gateways, that come back online?

Subscriptions are not persistent, so I somehow need to find all things (from that gateway), where the desired state != reported state and update them again.

Can there be a rule to do that? The idea is to have the gateway publish a reconnect message (like /reconnect/<gatewayId>) when it comes back online. The rule would have to find all device shadows for that gateway, where desired state != reported state and publish them.

NB: I know that I can program the mechanism without the device shadow with a database of its own. But the idea was to use the IoT mechanism for that.

Another question: Creating the bitmaps is really quick (could be a 1000 per second) , sending to the displays can be very slow (especially sending the first bitmap of a bunch can take a minute). So there might several thousand bitmaps (for one gateway) created before the first message can be confirmed. Is this a problem?

1
amazon-web-servicesaws-iot

1 Answers

4
votes

If I understand your use case correctly, I think there may be some changes needed to your concept to make it work better. I will try to answer your questions breaking them down into smaller parts.

  1. State Synchronization: Since your displays have no direct communication with AWS IoT, it would be better if you treated your gateways as things and every display as an attribute (e.g <display_id>) of the respective gateway thing. That way, whenever a new image has to be uploaded to a display, you can simply update the desired state of a nested attribute to the respective display attribute (e.g a <bitmap_version> nested to <display_id>). You can do that using the thing shadow UPDATE topic (e.g $aws/things/<gateway_id>/shadow/update). You can trigger a message to the UPDATE topic using Lambda to detect when a new version of the display bitmap has been uploaded to S3.

  2. Image Download: Whenever a new version of the bitmap has been uploaded to S3, the gateway receives the new desired state for the specific bitmap version of the display attribute via the thing's ACCEPTED UPDATE topic (e.g $aws/things/<gateway_id>/shadow/update/accepted), downloads the new bitmap, updates the display via your proprietary wireless protocol and updates the reported state of the attribute on the thing shadow UPDATE topic.

  3. Handling Disconnected Gateways: Yes, subscriptions are not persistent, but if you treat your gateways as things and every display as an attribute of that thing, whenever it comes back online it can post a message to the GET topic (e.g $aws/things/<gateway_id>/shadow/get), check the thing's current state on the ACCEPTED GET topic ($aws/things/<gateway_id>/shadow/get/accepted) and then proceed to download the new bitmaps in case of new versions.

  4. Handling Large Data Volume: If you need to update every display of a gateway with several bitmaps per second, considering that you have thousands of displays per gateway, I think the problem you may run into is a bandwidth bottleneck and synchronizing all of those MQTT messages with your thing shadow topics. If you only need to update each display once in a while, I think your concept could work very well.

Some things to take into consideration:

  1. The AWS IoT MQTT implementation cannot guarantee the order in which messages are received. If you need your messages to be received in a specific order, you have to implement that on your application.

  2. AWS IoT is still a beta service, so many implementation details may change.