Changing bits in a frame in video using OpenCV

0
votes

Current scenario:

I am doing a project in Video processing which requires me to make modifications in the contents of individual frames of a video file.

My Approach:

  • Split the video into multiple frames i.e. RGB images.
  • Modify the images assuming RGB images
  • Replace the modified frames in place of the original frames.

My Questions:

  • What kind of video formats will allow me to do such processing?
  • Another way to put it is will lossy video formats like MP4 allow me to edit the image (in RGB format) and replace it safely?
1
opencvvideovideo-processing
for most lossy formats you'll change many frames if you change a single rgb image because an encoded frame typically depends on the content of the previous frame. - Micka
@Micka To ma knowledge he is not compressing to encode the data. He is just processing. Yeah i agree with you if he is encoding, he needs "P" and "B" frame to predict the block in the next frame. We can modify any required frame for processing. - Arjun

1 Answers

2
votes

If you're still open to ideas to explore...

These media container formats seems to allow including bitmap bytes as video frames.

  • MOV container (Apple Quicktime) seems to support a PNG codec

  • AVI container (supports RGBA lossless codecs - see Lagarith & HuffYUV)

  • FLV container (supports both bitmap data and JPEG) : The bitmap codec is referred to a "ScreenVideo" but its just uncompressed RGBA information wedged between some 11 front-bytes (defines frame/tag header inc timestamps etc) + your bitmap data + closing 4 bytes (an integer confirming the tag size).

  • MP4 only holds MPEG codecs (so must the picture must be H.264 or H.265 format) : Are you using C++? I think someone made a keyframe encoder for H.264 so maybe you can use that code to supply thr final encoding (or at least end up with .h264 file that can be put into MP4 by FFMPEG or such tool)

Basically use OpenCV to handle whatever format you have, stepping through frame by frame and extracting each frame to RGBA then edit the image grab and either put that bitmapdata to FLV or use that Cardinal Peak's H264 encoder code. You dont need an FLV encoder just some way to write bytes defining an FLV file. Its so easy you can even use a hex-editor (obviously pasting in the long bitmap data where required)