How to convert tensorflow image Tensor to Numpy array inside Dataset?

Question

I want to create some image mask during image augmentation. Example image

Code:

import tensorflow as tf
import cv2
import pandas

# You can replace to local image
train_df = pd.DataFrame({'image_id': ['https://i.stack.imgur.com/CMEaA.jpg'],  
                         'label': [1]})


def create_mask(image, label):
    print(type(image)) # <class 'tensorflow.python.framework.ops.Tensor'>
    if isinstance(image, str):
        img = cv2.imread(image)
    else:
        img = image

    ## convert to hsv
    hsv = cv2.cvtColor(img, cv2.COLOR_BGR2HSV)
    ## mask of green (36,0,0) ~ (70, 255,255)
    mask1 = cv2.inRange(hsv, (36, 0, 0), (70, 255,255))
    ## mask o yellow (15,0,0) ~ (36, 255, 255)
    mask2 = cv2.inRange(hsv, (15,0,0), (36, 255, 255))
    ## final mask and masked
    mask = cv2.bitwise_or(mask1, mask2)
    result = cv2.bitwise_and(img,img, mask=mask)
    return result, label

train_ds = tf.data.Dataset.from_tensor_slices((
    train_df.image_id.values,train_df.label.values))
train_ds = train_ds.map(create_mask)

As result I get error because we have a tensor in the "image":

hsv = cv2.cvtColor(img, cv2.COLOR_BGR2HSV)

TypeError: Expected Ptr<cv::UMat> for argument 'src'

Ok, we need a numpy array. But if I try img = image.numpy() I got error:

AttributeError: 'Tensor' object has no attribute 'numpy' As expected...

Also I tried eval() with sess.run() but got error for placeholder, something like "tensor unhashable, use tensor.ref()", but if I use ref(), I get something like "cannot use Tensor".

Well, I have one simple question - can anybody advice me a working way to convert Tensor to numpy array during image process inside tf.data.Dataset?

Nicolas Gervais Nicolas Gervais · Accepted Answer · 2021-01-05T12:55:01

Try using only Tensorflow functions. For example, you can use tf.image.rgb_to_hsv.

rgb = tfio.experimental.color.bgr_to_rgb(img)
hsv = tf.image.rgb_to_hsv(rgb)

You should find a Tensorflow way to do the following operations too.

How to convert tensorflow image Tensor to Numpy array inside Dataset?

1 Answers