How to integrate mlflow with tensorflow object detection api

Question

I am trying to use the mlflow.tensorflow.autolog() with tensorflow object detection api. Adding mlflow.tensorflow.autolog() to model_main.py logs some parameters like global_norm/clipped_gradient_norm,global_norm/gradient_norm,global_step/sec,learning_rate_1,loss_1,loss_2 in mlflow. However the more important metrics like map,precision,recall are not being logged in mlfow.

Answer 1

The mlflow autologging will log whatever metrics are specified for the keras model, e.g. as in https://keras.io/metrics/ and https://github.com/mlflow/mlflow/blob/master/examples/keras/train.py#L66. Are those metrics specified but not logged?

Answer 2

Did you figure out this? I am doing the same project. I added

import mlflow.tensorflow
mlflow.tensorflow.autolog()

in the beginning of model_main_tf2.py file but couldn't see any metrics recorded in the MLFlow. May I ask where did you put the mlflow code in the model_main_tf2.py? I also tried to add the above code in the model_lib_v2.py file but no luck.

I installed tensorflow object detection in the cluster and run the training with shell command to tun the training:

%sh

cp /dbfs/mnt/s3bucket/path/model_lib_v2.py /opt/tensorflow/models/research/object_detection/model_lib_v2.py
cp /dbfs/mnt/s3bucket/path/model_main_tf2.py /opt/tensorflow/models/research/object_detection/model_main_tf2.py

python /opt/tensorflow/models/research/object_detection/model_main_tf2.py --logtostderr --model_dir=$HOME/object_detection/mandrel --pipeline_config_path=/dbfs/mnt/s3bucket/path/model/efficientdet_d0_coco17_tpu-32/pipeline.config --num_train_steps=600

Thanks.