Are there any way to find optical flow velocity from motion of real-world object without knowing depth Z?

9
votes

I am struggling and need help.

I want to compute optical flow velocity from the known motion of real-world object (actually camera is moving). This is part of what I have asked in my previous question (Determining if a feature is part of a moving object from sparse optical flow (KLT)).

Anyway, I have done computing optical flow using cvGoodFeaturesToTrack() and cvCalcOpticalFlowPyrLK().

I just want to check if the flow I computed is theoretically correct (corresponding to the motion of camera).

Let my camera move only in Z axis (simply ignore yaw-rate for now). Assume my camera move for Vz (in Z direction).

I can find the optical flow by

vx = x * Vz / Z    
vy = y * Vz / Z

(assume Vx,Vy = 0 --> no camera motion in x and y axis)

This is what I have studied mainly from http://www.cse.psu.edu/~rcollins/CSE486/lecture22_6pp.pdf.

The problem is to solve this I have to have Z. In my case, I cannot assume surface Z to be flat or known. Camera is moving on road and directing perpendicular to the ground.

Please anyone help me answer the questions below:

  1. How can I get the depth of object Z value? Do I need additional technique?
  2. Or are there any way else that I can find the relationship between motion of camera and image optical flow?
  3. Have anyone tried the equation above? Is it valid when the camera moving just in one direction?

Thank you very much.

[If you find this question to be too vague, please let me know so that I can give more detail.]

1
opencvcomputer-vision
If you have multiple cameras you could use stereo vision to determine the distance. If you have a reference object with known distance, you could calculate the parallax of another object and use that to find its distance. Other than that I'm afraid the distance of arbitrary objects might be rather hard if not impossible to determine. – Junuxx
Thank you for your comments. My question is for single camera and I dont have any knowledge about other reference objects. I have tried on multiple images and assumed optical flow from previous pairs of images to be reliable and used them to compute Z. Anyway, I found this also a problem as well. – Sonia
You can in theory get the Z from estimating the fundamental matrix using an 8-point or 5-point method, and decomposing the fundamental matrix, but as they say "the path is fraught with danger". It is probably also a much more complex solution to the very problem you are trying to solve with optical flow. – dvhamme
Although you can do it computing the fundamental matrix and decomposing it into rotation and translation, it is always up to scale, which means that the scale you get for different pairs of consecutive images is different, so that you can not get the "true" movement of the camera that way. The easiest solution is to consider only optical vectors of a dominant plane (the ground). If not, inference methods, like particle filters (see Monoslam code.google.com/p/monoslam) can do the work. – marcos.nieto

1 Answers

3
votes

Perhaps this could help... Video lectures from University of Central Florida Computer Vision Group:

Additional python codes from Jan Erik Solem: Programming Computer Vision with Python.

Read chapter 10.4, it will most probably answer all your questions.

Also look at chapter 5.4 of that book, if you take an image with the camera and then move your camera slightly in the x-direction and take another image you can calculate something called a "disparity map" using the two images which tells what kind of things in the image were in the front vs back. this is slightly like figuring out the z-direction. something along the lines of what you have already tried and what some of the comments had mentioned about stereo imaging.

Chapter 4.3 explains pose estimation using planar markers. You can use object placed in front of the camera at a known distance to calibrate the camera. This is most probably what you should look at first.