Compute road plane normal with an embedded camera

Question

I am developing some computer vision algorithms for vehicle applications. I am in front of a problem and some help would be appreciated.

Let say we have a calibrated camera attached to a vehicle which captures a frame of the road forward the vehicle: Initial frame

We apply a first filter to keep only the road markers and return a binary image: Filtered image

Once the road lane are separated, we can approximate the lanes with linear expressions and detect the vanishing point: Objective

But what I am looking for to recover is the equation of the normal n into the image without any prior knowledge of the rotation matrix and the translation vector. Nevertheless, I assume L1, L2 and L3 lie on the same plane.

In the 3D space the problem is quite simple. In the 2D image plane, since the camera projective transformation does not keep the angle properties more complex. I am not able to find a way to figure out the equation of the normal.

Do you have any idea about how I could compute the normal?

Thanks, Pm

Answer 1

No can do, you need a minimum of two independent vanishing points (i.e. vanishing points representing the images of the points at infinity of two different pencils of parallel lines).

If you have them, the answer is trivial: express the image positions of said vanishing points in homogeneous coordinates. Then their cross product is equal (up to scale) to the normal vector of the 3D plane said pencils define, decomposed in camera coordinates.

Answer 2

Your information is insufficient as the others have stated. If your data is coming from a video a common way to get a road ground plane is to take two or more images, compute the associated homography then decompose the homography matrix into the surface normal and relative camera motion. You can do the decomposition with OpenCV's decomposeHomographyMatmethod. You can compute the homography by associating four or more point correspondences using OpenCV's findHomography method. If it is hard to determine these correspondences it is also possible to do it with a combination of point and line correspondences paper, however this is not implemented in OpenCV.

Answer 3

You do not have sufficient information in the example you provide.

If you are wondering "which way is up", one thing you might be able to do is to detect the line on the horizon. If K is the calibration matrix then K^Tl will give you the plane normal in 3D relative to your camera. (The general equation for backprojection of a line l in the image to a plane E through the center of projection is E=P^Tl with a 3x4 projection matrix P)

A better alternative might be to establish a homography to rectify the ground-plane. To do so, however, you need at least four non-collinear points with known coordinates - or four lines, no three of which may be parallel.