Combine two different device capture frame into one frame and write a video

Question

I'm beginner in opencv but well versed in C#/C++. I have created a openCV console app to capture and write frames data to videos from multiple devices or RTSP streams which is working fine. I require to merge the output of two separate devices frames into one frame and write to video have done this in the snippet below but the video file generated is corrupted.

The video capture and video writer are set to grab at 1920 x 1080 resolution.

  for (int i = 0; i < devices; i++) {
  Mat frame;
  Mat f1, f2;
  //Grab frame from camera capture
  if (videoCapturePool[i].grab()) {
    videoCapturePool[i].retrieve(f1);
    f2 = f1.clone();
    if (isPause) {
      circle(f1, Point(f1.size().width / 2, f1.size().height / 2), 10, (255, 255, 10), -1);
      circle(f2, Point(f2.size().width / 2, f2.size().height / 2), 10, (255, 255, 10), -1);
    }
    hconcat(f1, f2, frame);

    imshow("Output", frame);

    waitKey(10);

    frameData[i].camerFrames.push_back(frame);
  }
  f1.release();
  f2.release();

}

for (int i = 0; i < devices; i++) {
  int32_t frame_width(static_cast < int32_t > (videoCapturePool[i].get(CV_CAP_PROP_FRAME_WIDTH)));
  int32_t frame_height(static_cast < int32_t > (videoCapturePool[i].get(CV_CAP_PROP_FRAME_HEIGHT)));

  VideoWriter vidwriter = VideoWriter(videoFilePaths[i], CV_FOURCC('M', 'J', 'P', 'G'), 30, Size(frame_width, frame_height), true);

  videoWriterPool.push_back(vidwriter);
  writer << frameData[i].camerFrames.size() << endl;
  for (size_t j = 0; j < frameData[i].camerFrames.size(); j++) {
    //write the frame
    videoWriterPool[i].write(frameData[i].camerFrames[j]);
  }
  frameData[i].camerFrames.clear();
}

Nuzhny Nuzhny · Accepted Answer · 2019-03-14T07:10:28

Your approach must be generally refactored. I don't say about code - just about architecture. It has a big problem with memory - let's calculate!

The size of the one RGB frame 1920x1080: frame_size = 1920 * 1080 * 3 = 6 Mb
How many frames do you want capture from 2 cameras? For example 1 minute of video with 30 fps: video_size = 2 camera * 6 Mb/frame * 60 seconds = 21 Gb! Do you have so memory per process?

I advice to make queues in different threads for capture frames and 2 threads for pick up frames from capture queues and write it to files.

Combine two different device capture frame into one frame and write a video

1 Answers