output from solvePnP doesn't match projectPoints

0
votes

I get strange data from solvePnP, so I tried to check it with projectPoints:

retval, rvec, tvec=cv2.solvePnP(opts, ipts, mtx, dist, flags=cv2.SOLVEPNP_ITERATIVE)
print(retval,rvec,tvec)
proj, jac = cv2.projectPoints(opts, rvec, tvec, mtx, dist)
print(proj,ipts)

here opts are 3d points with z=0, detected on this picture:

enter image description here

And ipts are taken from this pic (only part of picture here):

enter image description here

I've checked points themselves (detected with SIFT, points are detected correctly and pairing in a right way).

Now I want to test if rvec and tvec, found by SolvePnP is correct, so I invoke cv2.projectPoint to test if 3d points are projected to the image points. And here is what I have:

enter image description here

So I see that projected points lie outside of image, having y<0.

(retval from solvePnP is true)

This is distortion matrix dist: 

1.6324642475694839e+02 -2.1480843988631259e+04 -3.4969507980045117e-01 7.9693609309756430e-01 -4.0684056606034986e+01

This is mtx:

6.4154558230601404e+04 0. 1.2973531562160772e+03
0. 9.8908265814965678e+04 9.5760834379036123e+02
0. 0. 1.

This is opts:

[[ 1708.74987793  1138.92041016     0.        ]
 [ 1708.74987793  1138.92041016     0.        ]
 [ 1708.74987793  1138.92041016     0.        ]
 [ 1708.74987793  1138.92041016     0.        ]
 [ 1708.74987793  1138.92041016     0.        ]
 [ 1708.74987793  1138.92041016     0.        ]
 [ 1708.74987793  1138.92041016     0.        ]
 [ 1984.09973145  1069.31677246     0.        ]
 [ 1984.09973145  1069.31677246     0.        ]
 [ 1908.19396973  1200.05529785     0.        ]
 [ 1994.56677246  1286.16516113     0.        ]
 [ 1994.56677246  1286.16516113     0.        ]
 [ 1806.82177734  1058.06872559     0.        ]
 [ 1925.55639648  1077.33703613     0.        ]
 [ 1998.30627441  1115.51647949     0.        ]
 [ 1998.30627441  1115.51647949     0.        ]
 [ 1998.30627441  1115.51647949     0.        ]
 [ 2013.79003906  1168.08728027     0.        ]
 [ 1972.93457031  1234.92614746     0.        ]
 [ 2029.11364746  1220.234375       0.        ]]

This is ipts:

[[  71.6125946    11.61344719]
 [ 116.60684967   71.6068573 ]
 [ 116.60684967   71.6068573 ]
 [ 101.60684967   86.60684967]
 [ 101.60684967   86.60684967]
 [ 116.60684967  101.6068573 ]
 [ 116.60684967  101.6068573 ]
 [ 112.37421417   53.40462112]
 [ 112.37421417   53.40462112]
 [  83.76233673   84.36077118]
 [  98.45358276  112.38414764]
 [  98.45358276  112.38414764]
 [  67.2594223    38.04878998]
 [  96.85155487   51.85028076]
 [ 112.26165009   67.25630188]
 [ 112.26165009   67.25630188]
 [ 112.26165009   67.25630188]
 [ 112.24694061   82.24401855]
 [  96.82528687   97.66513824]
 [ 112.2511673    97.25905609]]

rvec = [[-0.21890167] [-0.86241377] [ 0.96051463]]
tvec = [[  239.04461181] [-2165.99539286] [-1700.61539107]]

Also I tried to follow one of the comments and multiply each y from opts by -1, but this gave me even more crazy coordinates outside picture like 10^13.

1
pythonopencvaugmented-realityopencv-solvepnp
If you can provide all of your data i.e. points and calibration matrices, we can check it out. – yapws87
3D points are expected in object space i.e. X axis rightward, Y axis upward and Z axis backward (Z=0) whilst 2D points are expected in image space i.e. X axis rightward and Y axis downward. Y axes are different. Finally, the object rotation + position from camera are computed in camera space i.e. X axis rightward, Y axis downward and Z axis forward. – RCYR
Does it mean I have to multiply y by -1 before projecting? – Stepan Yakovenko

1 Answers

2
votes

Camera matrix (mts) is incorrect. Fx and Fy are very different (Fx=6.4154558230601404e+04 Fy=9.8908265814965678e+04) and very big. According to comment in OpenCV calibrateCamera() function this problem usually occurs because you have probably used patternSize=cvSize(rows,cols) instead of using patternSize=cvSize(cols,rows) in findChessboardCorners.