KairoDB metric names or tags for IoT

Question

We have a lot of sensors like energymeters and want to store the data using kairosdb. Before we used a simple SQL store, where each sensor has it's own table where each measurement is one row.

a measturement is published to the system from the sensor via a JSON object:

{
    time: 12363453,
    volt: 238.33,
    ampere: 9.3,
    watts: 29.0,
}

So, for example for two sensors on two devices, we have this in our DB:

Sensor A1234:

id, time, volt, ampere, watts
12, <unix-ts>, 238.33, 9.3, 29.0
13, <unix-ts>, 238.21, 9.1, 23.8
...

Sensor B5678:

id, time, volt, ampere, watts
75, <unix-ts>, 230.12, 3.9, 19.5
76, <unix-ts>, 234.65, 2.8, 24.5
...

Now, we're investigating what's the best solution to store the same information but using KairosDB instead.

Our goal is to answser some "questions" like: - the latest volt, watt, ampere - what was the average ampere between x (start timestamp) and y (end timestamp) - what was the sum watts over all (or a subset) sensors between two dates

and so on.

So, what would be the best approach for choosing metric names and/or tags?

Should we use the sensor-names for the metric names (without tags):

sensors.energy.a1234.volt=238.33
sensors.energy.a1234.ampere=9.3
sensors.energy.a1234.watts=29.0

sensors.energy.b5678.volt=230.12
sensors.energy.b5678.ampere=3.9
sensors.energy.b5678.watts=19.5

Or should we use tags on the same metrics for all sensors:

sensors.energy.volt=238.33, tag: sensor=a1234
sensors.energy.ampere=9.3, tag: sensor=a1234
sensors.energy.watts=29.0, tag: sensor=a1234

sensors.energy.volt=230.12, tag: sensor=b5678
sensors.energy.ampere=3.9, tag: sensor=b5678
sensors.energy.watts=19.5, tag: sensor=b5678

or, are we totally on the wrong way?

Is there a difference regarding the query performance?

Loic Loic · Accepted Answer · 2018-04-26T08:45:05

it depends on your usage and in the tags cardinality.

If you need to group-by sensor it's useful to have in tag. If you always query sensors individually you will have simpler queries by having in the metric name.

If the tags cardinality (number of time series for a metric) is too high it affects query performances (I would saw at hundred thousands+). There are methods to overcome this impact by using an indexing/search plugin like the kairosDB SolR search (https://github.com/kairosdb/kairos-solr).

I hope this helps.

KairoDB metric names or tags for IoT

1 Answers