Tensorflow Object Detection: visualization_export_dir in eval_config not saving eval images

Question

As the title states, I would like to save the detection vs. ground truth images of the eval from Tensorboard.

From the eval proto:

// Path to directory to store visualizations in. If empty, visualization
// images are not exported (only shown on Tensorboard).
optional string visualization_export_dir = 6 [default=""];

Therefore I have added it to my model.config file as such. Keep in mind I am working from the /Object-Detection/ directory.

eval_config: {
  num_examples: 57

  visualization_export_dir: "bevelgear_training/eval_images/"
  num_visualizations: 57
  metrics_set: "pascal_voc_detection_metrics"
}

Training finishes as normal, I can see all the eval images in tensorboard BUT the /eval_images/ folder is empty.

Anyone get this working?

Answer 1

While the parameter is available in the config file, this feature is not actually supported. Instead, you can use TensorBoard and see the evaluated images with overlaid detections under images tab.

Answer 2

Your config looks correct, and the feature is implemented in current TensorFlow versions. This is my eval_config:

eval_config: {
  num_examples: 100
  num_visualizations: 20
  visualize_groundtruth_boxes: true
  visualization_export_dir: "F:/project_name/tf-eval/"
  include_metrics_per_category: true
}

I get visualisations in both TensorBoard and saved to file. It could be that your relative paths are causing an issue, although that normally is not a problem in the .config file. The export directory must exist - otherwise no images will be output.

Answer 3

append the lines :

    import matplotlib.pyplot as plt
    import os
    vutils.save_image_array_as_png(sbys_image[0], os.path.join(eval_config.visualization_export_dir, 'eval_{}_{}'.format(i,j) + ".png"))

after the code should look like :

  for j, sbys_image in enumerate(sbys_image_list):
    tf.compat.v2.summary.image(
        name='eval_side_by_side_{}_{}'.format(i, j),
        step=global_step,
        data=sbys_image,
        max_outputs=eval_config.num_visualizations)
    import matplotlib.pyplot as plt
    import os
    vutils.save_image_array_as_png(sbys_image[0], os.path.join(eval_config.visualization_export_dir, 'eval_{}_{}'.format(i,j) + ".png"))

at model_lib_v2.py line number 992

be careful, not to fix your cloned file. model_lib_v2.py should be in the path something looks like /python-3.6/pakages/object_detection/util....

thx