Storing key value pairs in Elastic and visualising with Kibana

Question

I have a set of key value pairs returned from an external API that I would like to store in Elastic and be able to visualize in Kibana.

The data is returned from the external API as a JSON object containing multiple key value pairs

{
    "value" : 
    {
        "en_GB": 181,
        "en_US": 86,
        "es_ES": 20,
        "fr_FR": 10
    }
}

I would like to be able to store this data in Elastic and create some charts in Kibana like a pie chart for example.

Currently I am taking this json, calculating each value as a percentage and storing it in a .NET dictionary, then adding this dictionary as a property on a larger object to be stored in Elastic as one document.

This works fine and I can manually create some charts in Kibana by selecting each field. This issue with it is that the keys/fields returned from the API will be different each time.

Is there a way in Kibana to dynamically select the fields to display in a graph, for example by selecting fields by their location in an object or by the parent object name etc?

Or am I going about this in the wrong way and need to restructure the data before sending it to Elastic?

I am using version 7.11.2

Any help is greatly appreciated, thanks.

Seb Christie Seb Christie · Accepted Answer · 2021-03-24T15:06:58

For anyone else who comes across this issue the solution I went with was to index each key value pair as its own document, something like

{ "key": "en_GB", "value": 100 }
{ "key": "en_US", "value": 90 }

If you need to you can then join these by an ID but this hasn't been necessary for me so far

Storing key value pairs in Elastic and visualising with Kibana

1 Answers