I am doing structure from motion from multiple images using OpenCV. I have 8 images, and I have generated 3D point clouds for each pair of images (img1&2, img2&3, img3&4 etc.). I know that each individual 3D point cloud is correct because they look good when displayed in VTK / OpenGL.
My cameras are (roughly) calibrated, using EXIF metadata for the focal length and the center of the image as the principal point.
How do I transform each of these 3D point clouds into the 3D coordinate system of the leftmost camera?