Using a depth map

1
votes

I have had issues finding any information on how to use a depth map/image to gain the distance to an obstacle.

TLDR: I have a depth map as an OpenCV Mat, I know it's CV_16UC1, but I dont know how to get the distances from it.

I have an Intel Realsense D415 camera, I have installed the SDK, the ROS wrapper, I have a topic with a depth map published. (/camera/depth/image_rect_raw)

Next, I've written a little program in C++ that converts the image using cv_bridge to an OpenCV image and I display it in a window, so I know it's working. (image: https://i.imgur.com/QyKWp2J.png )

Now I need to get the distance from it, which I have no idea how to do and have been unsuccessful in finding help for. I imagine I'm only going to want to use the top half or 2/3 of the image, because of the way the camera will be mounted, so the bottom third/half will only contain of floor/ground.

I feel like I am missing something big and simple in order to make this work, but I literally dont even know what to do now.

1
c++opencvros
I depth map usually stores the depth (= distance) for each pixel from the viewpoint (camera). In order to get the distance, you simply have to read a single depth value (RGB) for a given pixel. - dan
See this example: rs-pointcloud.cpp. If you want to know how compute the pointcloud from the depth map, have a look at: pointcloud.cpp. - Catree
The issue I've had was that I do not have access to the camera API (=librealsense2), I only got the image feed. Turns out I really just could get value of pixel (x,y) and get the distance in mm. I just had to know it was an u_int16_t matrix. - Welsy

1 Answers

0
votes

If you have a depth map and you want to generate 3D points from it, you can use reprojectImageTo3D() function of OpenCV.

Before that, you need to have the disparity-to-depth mapping matrix, which can be obtained using stereoRectify() function.