Fiware IoT Agent - data loss issue

Question

We are observing significant data loss in IoT Agent. The data loss happens when IoT agent receives data from our VerneMQ MQTT broker.

Our data is flowing at a rate of 1000/min. When the IoT Agent is started, it works fine for approximately 12 hours, but after 12 hours we see average data loss of 20%.

We have one other subscriber apart from IoT Agent which receives the same data from VerneMQ topic, and that subscriber receives all the data from VerneMQ. The data loss is only in IoT Agent.

We are running IoT Agent and Orion Context Broker on AWS environment using ECS/Fargate services. Each IoT Agent and Orion instance runs in a separate dedicated container.

When IoT Agent is running on one ECS container it works fine without data loss, but when ECS container increased to more than one, there seems to be a data loss. Can you please guide and help us to resolve this issue.

Details about our environment:

IoT Agent version:

{ "libVersion": "2.12.0-next", "port": "4041", "baseRoot": "/", "version": "1.14.0-next" }

Orion Context Broker version:

{   "orion": {
"version": "2.3.0",
"uptime": "0 d, 3 h, 51 m, 36 s",
"git_hash": "764f44bff1e73f819d4e0ac52e878272c375d322",
"compile_time": "Tue Nov 5 09:38:37 UTC 2019",
"compiled_by": "root",
"compiled_in": "38ab37448d3a",
"release_date": "Tue Nov 5 09:38:37 UTC 2019",
"doc": "https://fiware-orion.rtfd.io/en/2.3.0/"   } }

Environment variables set in IoT Agent:

fgalan fgalan · Accepted Answer · 2020-07-30T10:00:38

Taking into account:

When IoT Agent is running on one ECS container it works fine without data loss, but when ECS container increased to more than one, there seems to be a data loss

it seems the problem in somehow related with the underlying infrastructure but not with the FIWARE software itself. Thus, I'd suggest to review your AWS ECS settings (unfortunatelly, I'm not an expert in AWS ECS, so I cannot provide more specific feedback).

Fiware IoT Agent - data loss issue

2 Answers