Use case for Apache NiFi?

2
votes

I was wondering if the following IoT problem/use case would be an intended fit for using Apache NiFi:

I use NB-IoT/LTE-M as connectivity means for sending messages to an IoT cloud platform (e.g. AWS IoT Core, Azure IoT Hub or others). I need a protocol converter/gateway for the messages entering as UDP or TCP and leaving as MQTT. Of course I can develop an UDP/TCP listener/server that listens for the entering messages and publishing them to the desired IoT cloud platform (MQTT) broker. But I was thinking of eventual using Apache NiFi, as it has processors for UDP, TCP and MQTT. However, I was wondering if Apache NiFi is meant for these kind of (IoT) scenarios?

Thanks. Guy

1
apache-nifi

1 Answers

1
votes

We are using Apache NiFi to ingest and route IoT data at scale. I had to write a custom processor because of a proprietary IoT protocol, however assembling the rest of the flow has been just drag and drop. Before you invest into developing your own UDP/TCP listener/server at least try NiFi and see if you can solve your problem. With NiFi you can design your directed graphs of data routing pretty fast and have a short learn feedback loop.

Further think about:

  • What will limit the ability to grow the system?
  • Which resource constraints are important to pay attention to? E.g. metric volume, velocity, variety, volatility
  • How big can it get? Do you need resiliency?

With clustered NiFi you can spread your workload to multiple instances and satisfy the growth and resiliency requirement. You can also merge data and throttle its volume to protect downstream systems. The capabilities of NiFi are very versatile.

To answer your question: yes, Apache NiFi is actively used for IoT scenarios. There is even a NiFi IoT tutorial on cloudera: https://www.cloudera.com/tutorials/nifi-in-trucking-iot.html