New Tango 3D Reconstruction API

Question

I use the new 3D reconstruction API's (MIRA release). I have a problem when a call the Tango3DR_update function. It returns TANGO_3DR_INVALID code when I set the parameters associated with an image camera (const Tango3DR_ImageBuffer * image * const Tango3DR_Pose image_pose, Tango3DR_CameraCalibration const * calibration). I have checked my parameters, they seem to be correct. When I call this function without image parameters, this to work properly ... Is this a known bug?

thank you in advance for your answers.

Have you solved this problem? I get the same error, if I add the image buffer instead of a nullptr. — bashbug
Hi Alex, could you attach a logcat here, we can look into the file to find out more stuff. — xuguo
I tried to use the create function as follow Tango3DR_ImageBuffer* color_image = Tango3DR_ImageBuffer_create(1280, 720, TANGO_3DR_HAL_PIXEL_FORMAT_YCrCb_420_SP); color_image->stride = yuv_buffer_->stride; color_image->timestamp = yuv_buffer_->timestamp; memcpy(color_image->data, yuv_buffer_->data, 720*3/2*1280); Now I'm getting: E/tango: image_buffer.cc:27 Unknown pixel format. Maybe this points in the right direction? — bashbug

Julien Julien · Accepted Answer · 2016-06-22T11:07:19

TLDR; The support library ImageBufferManager has a bug with strides. Do color_image.stride = image_buffer->width; when creating your Tango3DR_ImageBuffer.

I think there are two things :

Image Format

First, you have to make sure to use the TANGO_HAL_PIXEL_FORMAT_YCrCb_420_SP. You can do that by using the ImageBufferManager from the support library.

ImageBufferManager and strides

Second, there is a catch if you use the support library ImageBufferManager though. TangoSupport_getLatestImageBuffer seems to fail to initialize the stride of the returned image (I got 0 and some other very large values) which the 3DR library doesn't like. The original TangoImageBuffer from OnColorAvailable has stride=1280 (=image_width) and forcing that value on the TangoImageBuffer returned from the ImageBufferManager seems to fix the issue. I believe this is a bug in ImageBufferManager.

This means doing

  color_image.stride = image_buffer->width;

instead of

  color_image.stride = image_buffer->stride

when creating the Tango3DR_ImageBuffer.

Full code example

I got it working with the following code in my Render method :

  TangoImageBuffer* image_buffer;
  ret = TangoSupport_getLatestImageBuffer(
      image_buffer_manager_, &image_buffer);
  if (ret != TANGO_SUCCESS) {
    LOG(ERROR) << "Error in TangoSupport_getLatestImageBuffer";
  }

  ...

  Tango3DR_ImageBuffer color_image;
  color_image.width = image_buffer->width;
  color_image.height = image_buffer->height;
  // VERY Important - The support library ImageBufferManager seems to have
  // a bug where it will always put the stride of the returned buffer
  // at 0, which causes 3DR to fail
  color_image.stride = image_buffer->width;
  color_image.timestamp = image_buffer->timestamp;
  color_image.format = (Tango3DR_ImageFormatType)image_buffer->format;
  color_image.data = image_buffer->data;

  ret = Tango3DR_update(
    tango_3dr_context_,
    &cloud,
    &depth_pose_3dr,
    &color_image,
    &color_pose_3dr,
    &tango_3dr_calibration_,
    &updated_indices);

I am using the ImageManager from the support library. So my OnColorAvailable looks like that

void SynchronizationApplication::OnColorAvailable(
    const TangoImageBuffer* buffer) {
  if (tango_3dr_enabled_ && tango_3dr_use_color_) {
    TangoErrorType ret = TangoSupport_updateImageBuffer(
      image_buffer_manager_, buffer);
    if (ret != TANGO_SUCCESS) {
      LOG(ERROR) << "Error in TangoSupport_updatePointCloud";
    }
  }
}

And the image_buffer_manager_ is initialized as follow (the pixel format might be important).

  TangoSupport_createImageBufferManager(
      TANGO_HAL_PIXEL_FORMAT_YCrCb_420_SP,
      image_width_,
      image_height_,
      &image_buffer_manager_
  );

I am copying the calibration as follow :

void CopyCalibrationTangoTo3DR(const TangoCameraIntrinsics& tango,
                               Tango3DR_CameraCalibration* out) {
  out->calibration_type =
    (Tango3DR_TangoCalibrationType)tango.calibration_type;
  out->cx = tango.cx;
  out->cy = tango.cy;
  memcpy(out->distortion, tango.distortion, sizeof(double) * 5);
  out->fx = tango.fx;
  out->fy = tango.fy;
  out->height = tango.height;
  out->width = tango.width;
}

New Tango 3D Reconstruction API

1 Answers