Spring Boot 2 Actuator doesnt publish jvm metric

Question

I am running a Spring Boot 2 Application and added the actuator spring boot starter dependency. I enabled all web endpoints and then called:

http://localhost:8080/actuator/metrics

result is:

{
    "names": ["jdbc.connections.active", 
              "jdbc.connections.max", 
              "jdbc.connections.min", 
              "hikaricp.connections.idle", 
              "hikaricp.connections.pending", 
              "hikaricp.connections", 
              "hikaricp.connections.active", 
              "hikaricp.connections.creation", 
              "hikaricp.connections.max", 
              "hikaricp.connections.min", 
              "hikaricp.connections.usage", 
              "hikaricp.connections.timeout", 
              "hikaricp.connections.acquire"]
}

But I am missing all the JVM stats and other built-in metrics. What am I missing here? Everything I read said that these metrics should be available at all times.

Thanks for any hints.

Answer 1

I want to share the findings with you. The problem was that a 3rd party library (Shiro) and my configuration for it. The bean loading of micrometer got mixed up which resulted in a too late initialisation of a needed PostProcessingBean which configures the MicroMeterRegistry (in my case the PrometheusMeterRegistry).

I dont know if its wise to do the configuration of the Registries via a different Bean (PostProcessor) which can lead to situations i had... the Registries should configure themselves without relying on other Beans which might get constructed too late.

Answer 2

In case this ever happens to anybody else: I had a similar issue (except it wasn't Graphite but Prometheus, and I was not using Shiro).

Basically I only had Hikari and HTTP metrics, nothing else (no JVM metrics like GC).

I banged my head on several walls before finding out the root cause: there was a Hikari auto configure post processor in Spring Boot Autoconfigure that eagerly retrieved a MeterRegistry, so all Metric beans didn't have time to initialize before.

And to my surprise, when looking at this code in Github I didn't find it. So I bumped my spring-boot-starter-parent version from 2.0.4.RELEASE to 2.1.0.RELEASE and now everything works fine. I correctly get all the metrics.

Answer 3

As I expected, this problem is caused by the loading order of the beans.

I used Shiro in the project.

Shiro's verification method used MyBatis to read data from the database.

I used @Autowired for MyBatis' Mapper file, which caused the Actuator metrics related beans to not be assembled by SpringBoot (I don't know what the specific reason is).

So i disabled the automatic assembly of the Mapper file by manual assembly.

The code is as follows:

public class SpringContextUtil implements ApplicationContextAware {

    private static ApplicationContext applicationContext;

    public void setApplicationContext(ApplicationContext applicationContext)
            throws BeansException {
        SpringContextUtil.applicationContext = applicationContext;
    }

    public static ApplicationContext getApplicationContext() {
        return applicationContext;
    }

    public static Object getBean(String beanId) throws BeansException {
        return applicationContext.getBean(beanId);
    }
}

Then

StoreMapper userMapper = (UserMapper) SpringContextUtil.getBean("userMapper");
UserModel userModel = userMapper.findUserByName(name);

The problem can be solved for the time being. This is just a stopgap measure, but at the moment I have no better way.

Answer 4

I can not found process_update_seconds in /actuator/prometheus, so I have spent some time to solve my problem.

My solution:

Rewrite HikariDataSourceMetricsPostProcessor and MeterRegistryPostProcessor;

The ordered of HikariDataSourceMetricsPostProcessor is Ordered.HIGHEST_PRECEDENCE + 1;

package org.springframework.boot.actuate.autoconfigure.metrics.jdbc;
...

class HikariDataSourceMetricsPostProcessor implements BeanPostProcessor, Ordered {

    ...

    public int getOrder() {
        return Ordered.HIGHEST_PRECEDENCE + 1;
    }
}

The ordered of MeterRegistryPostProcessor is Ordered.HIGHEST_PRECEDENCE;

package org.springframework.boot.actuate.autoconfigure.metrics;
...
import org.springframework.core.Ordered;

class MeterRegistryPostProcessor implements BeanPostProcessor, Ordered {
    ...
    @Override
    public int getOrder() {
        return Ordered.HIGHEST_PRECEDENCE;
    }
}

In my case I have used shiro and using jpa to save user session id. I found the order of MeterRegistryPostProcessor and HikariDataSourceMetricsPostProcessor cause the problem. MeterRegistry did not bind the metirc because of the loading order.

Maybe my solution will help you to solve the problem.

Answer 5

I have a working sample with Spring Boot, Micrometer, and Graphite and confirmed the out-of-the-box MeterBinders are working as follows:

{
  "names" : [ "jvm.memory.max", "process.files.max", "jvm.gc.memory.promoted", "tomcat.cache.hit", "system.load.average.1m", "tomcat.cache.access", "jvm.memory.used", "jvm.gc.max.data.size", "jvm.gc.pause", "jvm.memory.committed", "system.cpu.count", "logback.events", "tomcat.global.sent", "jvm.buffer.memory.used", "tomcat.sessions.created", "jvm.threads.daemon", "system.cpu.usage", "jvm.gc.memory.allocated", "tomcat.global.request.max", "tomcat.global.request", "tomcat.sessions.expired", "jvm.threads.live", "jvm.threads.peak", "tomcat.global.received", "process.uptime", "tomcat.sessions.rejected", "process.cpu.usage", "tomcat.threads.config.max", "jvm.classes.loaded", "jvm.classes.unloaded", "tomcat.global.error", "tomcat.sessions.active.current", "tomcat.sessions.alive.max", "jvm.gc.live.data.size", "tomcat.servlet.request.max", "tomcat.threads.current", "tomcat.servlet.request", "process.files.open", "jvm.buffer.count", "jvm.buffer.total.capacity", "tomcat.sessions.active.max", "tomcat.threads.busy", "my.counter", "process.start.time", "tomcat.servlet.error" ]
}

Note that the sample on the graphite branch, not the master branch.

If you could break the sample in the way you're seeing, I can take another look.