Labels on Prometheus metrics when serving multiple models with Tensorflow Serving

2
votes

We are serving multiple TF models using the standard tensorflow/serving docker container. Following the docs, we enabled monitoring by supplying a 'prometheus' config. Metrics are emitted as expected. TFS does label a few metrics in a way that differentiates between the separate models it is serving, for example, it shows the model_path on model load attempts and latency:

# TYPE :tensorflow:cc:saved_model:load_attempt_count counter
:tensorflow:cc:saved_model:load_attempt_count{model_path="gs://disco-int-100-mlops/ml-sandbox-101-tagging/servables/models/1598020514",status="success"} 1
:tensorflow:cc:saved_model:load_attempt_count{model_path="gs://ml-sandbox-examples/HousingModel/2020-10-05T10:37:56/1",status="success"} 1
# TYPE :tensorflow:cc:saved_model:load_latency counter
:tensorflow:cc:saved_model:load_latency{model_path="gs://disco-int-100-mlops/ml-sandbox-101-tagging/servables/models/1598020514"} 19274403
:tensorflow:cc:saved_model:load_latency{model_path="gs://ml-sandbox-examples/HousingModel/2020-10-05T10:37:56/1"} 2938076

Most other TFS metrics do not have any labels at all. Is it possible to provide additional configuration parameters so the we get, for example, :tensorflow:core:graph_run_time_usecs_histogram_bucket metrics labelled for each served model?

1
tensorflow-serving

1 Answers

0
votes

Answering my own question, the simple answer is "no, there are no additional configuration parameters for the Prometheus metrics config". The code that generates the metrics endpoint output is here: https://github.com/tensorflow/serving/blob/master/tensorflow_serving/util/prometheus_exporter.cc

Looks like an opportunity to make a contribution to TFS...