Convert OpenCV Mat or Image to NumPy array for Tensorflow

0
votes

This question is about getting everything working in .Net for Tensorflow object detection. Because I am in .Net, I am using the wrapper EmguCV for my OpenCV calls. As for Tensorflow, there are many wrappers for that in .Net as well.

My question is generic though, may be .Net specific but I don't think so. From Emgu or OpenCV, I am using VideoCapture to read frames from a video source. Each frame is a Mat, which can also be easily converted to an OpenCV image. However, Tensorflow object detection requires some sort of image array, which is suspect is a NumPy array, as an input.

I have found many examples and sample code but most of them have to do with having an image such as a jpeg on the file system and reading that in. Not many have to do with having an OpenCV Mat or Image and converting that to an input array for Tensorflow.

Specifically for the Tensorflow wrapper I am using, Tensorflow.Net, it actually takes something called a FeedItem as seen here Tensorflow.Net and again, this example reads in an image file.

I have been trying to figure it out. I have tried to take the Mat and Image Data property, which is a byte array and convert it to NumPy array but it doesn't work. I get exceptions saying something about jagged arrays are not supported.

So, simply, anybody know how to correctly and efficient convert these types of image data to NumPy or something I can feed into Tensorflow input?

Thanks!

1
.netopencvtensorflowemgucv

1 Answers

0
votes

Jagged arrays - are, actually, arrays of arrays, where outer array contains nested arrays, allocated elsewhere in memory. What you need to do is allocate a flat continuous array of the same type, for example,

byte[] data = new byte[numberOfImages * width * height * 3];

and copy bytes from the Data property you have using CopyTo. (I am not familiar with EmguCV, there might be a more efficient method to do this).

Then convert data into NumPy array.