Constraining intrinsic parameters for multi-video camera bundle adjustment within ceres-solver

Question

I have a set of 6 video clips shot by 6 cameras atop a vehicle. I'm trying to use CERES to bundle adjust O(1.4K) frames from these 6 cameras. Presumably the intrinsic parameters for each of the 6 video cameras are constant to very good approximation. So I'm trying to figure out a reasonable way to incorporate this physical constraint in CERES' framework.

As a CERES newbie, I've tried searching previous bundle adjustment examples & questions. But surprisingly, I haven't found CERES-solver code examples where camera intrinsics are constrained due to their corresponding to a small number of physical cameras.

Starting with the simple_bundle_adjuster.cc example code, I first tried separating apart the extrinsic, intrinsic and 3D point variables within the double* parameters_ member of the BALProblem class. I hoped that it's possible to construct separate ResidualBlocks for video frames which would have distinct extrinsic and 3D point parameters but shared intrinsic parameters depending upon the video camera's physical ID.

Here is my slightly modified version of the Create() method from the simple_bundle_adjuster.cc example:

  static ceres::CostFunction* Create(
    int i, const double observed_u, const double observed_v)
  {
     if (use_common_intrinsics_flag)
     { 
        return (new ceres::AutoDiffCostFunction<
                PinholeReprojectionError, 2,
                n_extrinsic_params, 
                n_physical_cameras * n_intrinsic_params, 3>(
                new PinholeReprojectionError(i, observed_u, observed_v)));
     }...

Unfortunately, I haven't been able to find any way to share the common intrinsic parameters from the 6 physical video cameras which is consistent with CERES.

Q1: Does CERES require combining all extrinsic and intrinsic parameters + 3D points into one large Residual Block for each physical video camera? Is this possible given that the number of frames collected by a video camera is not generally known at compile time?

Q2: Can constancy of intrinsic parameters for each physical video camera be approximately enforced via Upper/Lower Parameter Bounds? Maybe some iterative approach where parameter limits are tightened might work.

Sameer Agarwal Sameer Agarwal · Accepted Answer · 2019-08-06T13:43:32

If all six cameras have known constant intrinsics. Then there is no reason to have parameter blocks for them, just create a CostFunction object that stores and applies the intrinsics.

If you would like to have them be variable but say with a strong prior, add a parameter block - one for each of the six cameras, and then use that as one of the parameter blocks to the cost function that computes the residual error.

The same idea applies if all six cameras have the same intrinsics, in which case just have have one parameter block.

Constraining intrinsic parameters for multi-video camera bundle adjustment within ceres-solver

1 Answers