tf.image.crop_and_resize with bbox given by another python function

Question

I'm trying to test an existing network on cropped images. I have a rather large dataset, so I'm using the tensorflow dataset API. I first created a dataset containing all the names of the images I'm interested in and then use the flat_map() function to map the dataset of image names to dataset of cropped image patches.

So, here is the question. I do not know how many patches will be generated for this image, I have another python function get_image_regions which returns a n-by-4 numpy array of the boxes.

So I want to use something like:

boxes = tf.py_func(get_image_regions, [im_path], [tf.float32])

to get the set of boxes and use that boxes as input to tf.image.crop_and_resize()

However, since the return value from py_func is of unknow shape and rank, it cannot be used as input to the crop_and_resize() function. Is there any other way to work around this?

Answer 1

You can tf.reshape() the tensor if you're sure what the shape will be like this:

boxes = tf.reshape( tf.py_func(get_image_regions, [im_path], [tf.float32]), [ n, 4 ] )

That will fix the shape for the rest of the graph and will allow you to feed the tensor to tf.image.crop_and_resize() but will throw an error if it is ever fed the wrong size data.