OpenCV encoding to H264

10
votes

I want to encode images to H264 video in OpenCV.

As part of my software for image tracking, I'm using VideoWriter in OpenCV 3.4 (64-bit) with Visual Studio 2017 C++. I use ffmpeg for manual encoding, and as it comes with OpenCV I assume this is the best option. My version of OpenCV indicates it has been compiled with ffmpeg, and I can verify it loads the opencv_ffmpeg340_64.dll.

Decoding H264 (AVC1) is absolutely fine, including when specifically using the ffmpeg API. But encoding anything other than MJPG or raw images doesn't work: VideoWriter.Open() mostly returns false, for some cases it only writes an empty or small header but won't write frames. I've tried not only the ffmpeg API, but also any available API. Redirecting console/debug output to intermediate window in VC doesn't give any messages from OpenCV or ffmpeg.

There are numerous posts on previous versions of OpenCV using FFmpeg, including (Cisco) OpenH264 library and difficulties using this. I've tried many options, installing codecs pack including H264, ffmpeg static version, Cisco openH264 library, setting paths etc and even tried the low level ffmpeg library functions such as cvCreateVideoWriter exposed in OpenCV, and using different video containers (AVI, MPG, MP4) as well as fourcc strings.

I can't believe in 2018 the great OpenCV with FFmpeg support is unable to encode anything but arcane MJPG or raw images. If it uses FFmpeg surely a significant set of functionality should be available?

Though I think this should work, my next best option would be using a separate ffmpeg library, which would ironically add another ffmpeg.dll to my solution I imagine. Any suggestion appreciated!

Code fragment (using any video encoding API):

VideoWriter writer;
int codec = VideoWriter::fourcc('X', '2', '6', '4');
writer.open("test.avi", codec, 30, colorImage.size(), true);
writer.write(colorImage);
writer.release();
1
c++opencvvideoffmpegvideo-encoding
Compile your ffmpeg with libx264 for encoding h264. Even though decoder is usually shipped from the box, encoder rarely is. – Dmitrii Z.
Thank you for the suggestion, though this is not using OpenCV as I wanted and would create an extra ffmpeg dll, effectively using another external library - unless you are referring to recompiling the opencv_ffmpeg library without breaking OpenCV compatibility? – Yost777
You first build libx264, than configure ffmpeg to use libx264 (ffmpeg is autotools project, so ./configure should give you a hint of what is used), than build ffmpeg, than configure opencv to use fresh built ffmpeg (opencv is built with cmake, so look at cmake output to see if it uses correct version), than build opencv. It would result in opencv_ffmpeg with support of H264 encoding. – Dmitrii Z.
I would say that is quite a challenge to do on windows (at least for me, more used to do programming on unix), but should be doable with good environment, so good luck on that. Maybe its worth to search for prebuilt libraries for windows. – Dmitrii Z.
Thanks. I can try to build OpenCV (I've tried before and failed) - using this recent guide: learnopencv.com/install-opencv3-on-windows I don't know how to include a custom ffmpeg library, options, let alone how to get openx264 in there. This seems like a major operation moving away from standard opencv with only a single encoder supported. It seems it would be much easier to use a separate library for encoding, such as avcodec or openh264 directly. – Yost777

1 Answers

11
votes

To answer my own question - thank you for the previous suggestions though - I've managed to create and write frames to H264 video now, without any recompiling needed. This is what I've used:

  • Using OpenCV version 3.4.1
  • In the same folder as the exe: opencv_ffmpeg341_64.dll (provided with OpenCV)
  • In the same folder: openh264-1.7.0-win64.dll (available from: https://github.com/cisco/openh264/releases)
  • Use OpenCV VideoWriter class, omitting a specific encoding API
  • Container: MP4
  • Codec fourcc string: "H264"

I formatted my machine (Windows 10 64bit) so I can't exclude any issues with potential codec-clashing - also I have not fully tested for other combinations. As this appears to be such a common problem (with many misconceptions) I hope this is useful.