Cannot set custom SEI using Media Foundation h264 encoder

Question

I have set up a MFSinkWriter that takes a RGB MFMediaType in input and generates a H.264 MFMediaType as output. I need to add SEI information for every MFSample that is encoded. using Windows Media Foundation it should be possible to do so using the attribute MF_MT_FORWARD_CUSTOM_SEI for the output MFMediaType.

When encoding, before passing the MFSample to the MFSinkWriter, I add the MFSampleExtension_Encryption_SEIData with a blob containing the custom info that I need. However, when inspecting the produced file the SEI field are empty. There is no documentation/sample on how to use MF_MT_FORWARD_CUSTOM_SEI or MFSampleExtension_Encryption_SEIData. Any suggestion?

Answer 1

... using Windows Media Foundation it should be possible to do so using the attribute MF_MT_FORWARD_CUSTOM_SEI...

... There is no documentation/sample on how to use MF_MT_FORWARD_CUSTOM_SEI or MFSampleExtension_Encryption_SEIData

No, this is impossible. Specifically because there is no documented behavior for this, and there is no undocumented behavior for this too. You can achieve the desired by not using Sink Writer: if you manage the encoder yourself and you have raw H.264 output then you can update the NAL units and amend them with additional SEI NAL units you need.