How do Micrometer custom metrics work for containers with few short invocations when using Cloud Run and Stackdriver

Question

I have been reading the Cloud Run documentation and parts of the Micrometer documentation, but I am yet to be convinced that serverless workflows for custom metrics are fully supported by Micrometer, or even how it is supposed to work.

The Cloud Run autoscaling documentation states that:

For example, when an instance has finished handling requests, the container instance may remain idle for a period of time in case another request needs to be handled. An idle container instance may persist resources, such as open database connections. However, for Cloud Run (fully managed), the CPU will not be available

The important takeaways being:

• The instance may remain idle for "a period of time" - unspecified how long.
• CPU will not be available.

The Micrometer Stackdriver documentation states that metrics are sent to stackdriver with a default interval of one minute:

The interval at which metrics are sent to Stackdriver Monitoring. The default is 1 minute.

Takeaway:

• The service should run for at least one minute before any metrics are sent to stackdriver?

I also had a look at the Stackdriver part of the Micrometer Java implementation but could not find any special handling of this use-case, or any information on how to handle it.

Questions

• What happens if a Cloud Run container is invoked once every 30 minutes, but the run time is only 30 seconds? Will any custom metrics be sent to Stackdriver?

• Is there any way to mitigate this?

  • Assuming that it is impossible to know the run time of an invocation
  • And you only want to pay for the minimum amount of execution time

Answer 1

In reality, when no request is processed, the instance is kept up with a very low percentage of CPU allowed (for keeping up some states, like DB connexion pooling or stuff like this).

By the way, you can have a try on your workflow, it's possible that the very low CPU available can be enough for sending metrics to stackdriver. However, the percentage of the CPU allowed on idle instances can change at any time and your process go down without any change on your side

Maybe that the good pattern is to perform an explicit metric send in your code and not rely on automatic call frequency, by using Micrometer or directly the Stackdriver API.