TangoImageBuffer RAW/YUV failed to match frame

Question

I use the OnFrameAvailable callback to sync rgb with depth data. If I just render the rgb point cloud everything is fine.

But if I do some image processing the camera throws the following exceptions:

E/camera-metadata: /home/ubuntu/jobs/redwood_internal/RedwoodInternal/Redwood/common/player-engine/src/camera-metadata.cc:56 RAW failed to match frame

or

E/camera-metadata: /home/ubuntu/jobs/redwood_internal/RedwoodInternal/Redwood/common/player-engine/src/camera-metadata.cc:56 YUV failed to match frame

And TangoImageBuffer has complete garbage values. Sometimes black pixels, or the buffer half-half old and new pixel data.

I tried to solve it by threading. Everytime I got a new point cloud, the extra image processing thread needs about 1 sec cpu time. And it helped a little. After a few seconds the same behavior happend.

The problem is that I can't debug the nativ code properly. The monitoring of android studio shows normal cpu and gpu usage.

I've seen that user guppy had this problem with the Leipniz tango version, but no solution was posted. So I hope maybe someone else has managed this problem? Or has any suggestions?

EDIT

The behaviour disappeared, after using the tango_support library to copy xyz and yuv buffers.

This is probably a timing problem. If too much time is being taken in the callback doing image processing then it may be having trouble synchronizing. Try to copy the data out and process it in another thread, allowing other onFrameAvailable callbacks to proceed (and ignore them while the data is processing). — xuguo
I just did a test with a 1second sleep in the OnFrameAvailable callback, it reproduce the exact same error. I think the best way is to process this in the long running thread (i.e render thread). The video overlay c++ example actually shows this, here's the callback function that copies out data: github.com/googlesamples/tango-examples-c/blob/master/… — xuguo

xuguo xuguo · Accepted Answer · 2016-02-19T21:48:44

The "YUV failed to match frame" is most likely caused by the callback thread is taking too much time to execute. In short, you shouldn't do heavy processing in the OnFrameAvailable callback. This also applies to all other Tango callbacks, i.e pose or depth callback.

The solution to this would be copying out the byte buffer data and process it in another thread, potentially, the render thread. In the tango-example-c video-overlay-jni-example, the application does a memcpy to copy the data from callback thread to the render thread, so the processing of the data would not block the callbacks keep coming. See this line.

TangoImageBuffer RAW/YUV failed to match frame

1 Answers