Modelling database for uptime within influxdb efficiently

Question

After having played around with collectd and InfluxDB for a while, I've realized that the uptime is stored each and every time as a single data point, e.g. leading to measurements looking like this:

1469552552940296000 localhost   uptime  426568
1469552931893217000 localhost   uptime  426947
1469552991889480000 localhost   uptime  427007
1469553051889521000 localhost   uptime  427067
1469553111890071000 localhost   uptime  427127
1469553171889512000 localhost   uptime  427187
1469553231889512000 localhost   uptime  427247

This seems inefficient to me, since it is kind of redundant. Given the last measurement, I can calculate all others, so why store them in the first place? I'm now looking into retention policies, but I'm not quite sure how to apply them here. What would be a good strategy for this kind of data?

I definitely want to have the information when the system was powered off available, so basically I want to store the "start" point along with the latest uptime_value. Everything in between is redundant.

Michael Desa Michael Desa · Accepted Answer · 2016-07-28T22:27:29

The correct thing to do here is to here is to use continuous queries and retention policies. I don't know that you'll be able to store only the first and last points, but you can definitely.

The continuous query will be used to downsample all of your data to a single point. The retention policy will be used to drop the old data.

It would look something like this

CREATE RETENTION POLICY myrp on mydb DURATION 1d REPLICATION 1

Then have something like the following continuous query

CREATE CONTINUOUS QUERY mycq on mydb BEGIN
  SELECT max(uptime) FROM mymeasurement GROUP BY time(10m), *
END

That being said, after compression, each of those points will take up less than 2.5 bytes on disk. I probably wouldn't worry too much about being very efficient.

Modelling database for uptime within influxdb efficiently

1 Answers