Elastic APM different index name

Question

As of a few weeks ago we added filebeat, metricbeat and apm to our dotnet core application ran on our kubernetes cluster. It works all nice and recently we discovered filebeat and metricbeat are able to write a different index upon several rules.

We wanted to do the same for APM, however searching the documentation we can't find any option to set the name of the index to write to.

Is this even possible, and if yes how is it configured?

I also tried finding the current name apm-* within the codebase but couldn't find any matches upon configuring it.

The problem which we'd like to fix is that every space in kibana gets to see the apm metrics of every application. Certain applications shouldn't be within this space so therefore i thought a new apm-application-* index would do the trick...

Edit

Since it shouldn't be configured on the agent but instead in the cloud service console. I'm having troubles to 'user-override' the settings to my likings.

The rules i want to have:

When an application does not live inside the kubernetes namespace default OR kube-system write to an index called apm-7.8.0-application-type-2020-07
All other applications in other namespaces should remain in the default indices

I see you can add output.elasticsearch.indices to make this happen: Array of index selector rules supporting conditionals and formatted string.

I tried this by copying the same i had for metricbeat and updated it to use the apm syntax and came to the following 'user-override'

output.elasticsearch.indices:
- index: 'apm-%{[observer.version]}-%{[kubernetes.labels.app]}-%{[processor.event]}-%{+yyyy.MM}'
  when:
    not:
      or:
        - equals:
            kubernetes.namespace: default
        - equals:
            kubernetes.namespace: kube-system

but when i use this setup it tells me:

Your changes cannot be applied
'output.elasticsearch.indices.when': is not allowed

Set output.elasticsearch.indices.0.index to apm-%{[observer.version]}-%{[kubernetes.labels.app]}-%{[processor.event]}-%{+yyyy.MM}
Set output.elasticsearch.indices.0.when.not.or.0.equals.kubernetes.namespace to default
Set output.elasticsearch.indices.0.when.not.or.1.equals.kubernetes.namespace to kube-system

Then i updated the example but came to the same conclusion as it was not valid either..

Are your APM Server outputing directly to elasticsearch? The index name is defined in the APM Server. — leandrojmp
Yes, @leandrojmp is right, the .NET Agent itself does not control the index name, this is done in the APM Server and the linked docs explains it. ps: I work on the .NET Agent. — gregkalapos
Ahh i see! We are using the elastic cloud solution. So we haven't set up the configuration of APM-server itself. Is there anyway to update this configuration? — Baklap4

Val Val · Accepted Answer · 2020-07-27T14:58:27

In your ES Cloud console, you need to Edit the cluster configuration, scroll to the APM section and then click "User override settings". In there you can override the target index by adding the following property:

output.elasticsearch.index: "apm-application-%{[observer.version]}-{type}-%{+yyyy.MM.dd}"

Note that if you change this setting, you also need to modify the corresponding index template to match the new index name.

Elastic APM different index name

Edit

1 Answers