Modeling event counts with different actions in InfluxDB

Question

I have a flow of events that I'm indexing into Elasticsearch, and I'd like to record metrics on them using InfluxDB for graphing in Grafana. The flow is something like this:

Event comes in, with a type, that I'd like to track. The event could be an add or delete Sometimes, due to certain conditions, I might want to skip events that would otherwise be added.
If the event is something to be added, I might need to index several related events, and I'm interested knowing how many of these related events there are as well.

There are a relatively small number of discrete types, and I would like to be able to graph total events as well as total number of adds deletes and skips and break these down by type and whether or not they are a primary event or a related event.

This is a Go service using go-kit/kit/metrics/influx, which models counters as a positive delta of counts per tagset, so something like this

counter := influxProvider.NewCounter("my_counter")
counter.Add(10)
counter.With("error", "false").Add(1)
counter.With("error", "true").Add(1)
counter.With("error", "false").Add(1)
counter.Add(50)

Would emit the following in line protocol on the next update:

my_counter count=60
my_counter,error=true count=1
my_counter,error=false count=2

What is a good way to model this? Is it better to have separate measurements for add delete and skip, or is it better to have one event measurement and tag on type, action, and related? Or is there a better way to do this altogether?

Yuri Lachin Yuri Lachin · Accepted Answer · 2019-11-18T09:17:22

I' vote for one measurement with tags on type, action and related. With small number of discrete types it won't add high cardinality to data but will allow you to group on any combination of tags in queries.

And you'll still have an option to create and fill one measurement per tag on demand any time later if needed or use continuous queries to fill them automatically.

With one measurement per tag there will be no easy way to go back to multi-tag measurement afterwards.

Modeling event counts with different actions in InfluxDB

1 Answers