Media Foundation MP4 Encoding: IMFSinkWriter doesn't accept PCM input

Question

I'm encoding H264 video using Windows Media Foundation. (I use it via C# using the MFNet Wrapper).

The video encoding works perfectly. I get out a MP4 with H264 video.

Now I try to add audio. I want to pass in uncompressed PCM from memory, and get out encoded AAC. I add the audiostream to my SinkWriter. öBut when calling SetInputMediaType, I get MF_E_INVALIDMEDIATYPE.

Why is that?

Here's my code:

IMFMediaType audioTypeOut, audioTypeIn, videoTypeOut, videoTypeIn;
IMFSinkWriter sinkWriter;

Check = MFCreateSinkWriterFromURL(outputFile.Replace(".avi",".mp4"), null, attributes, out sinkWriter); // the assignment to "Check" throws an exception if a method fails.

//[ ... setup video input & output - works perfectly ...]
Check = sinkWriter.AddStream(videoTypeOut, out videoStreamIndex);
Check = sinkWriter.SetInputMediaType(videoStreamIndex, videoTypeIn, null);

// Now setup Audio:
// Input PCM from memory
Check = MFExtern.MFCreateMediaType(out audioTypeIn);
Check = audioTypeIn.SetGUID(MFAttributesClsid.MF_MT_MAJOR_TYPE, MFMediaType.Audio);
Check = audioTypeIn.SetGUID(MFAttributesClsid.MF_MT_SUBTYPE, MFMediaType.PCM); // this .PCM enum value is equal to MFAudioFormat_PCM in the C++ api (I checked)
Check = audioTypeIn.SetUINT32(MFAttributesClsid.MF_MT_AUDIO_NUM_CHANNELS, 2);
Check = audioTypeIn.SetUINT32(MFAttributesClsid.MF_MT_AUDIO_SAMPLES_PER_SECOND, 44100);
Check = audioTypeIn.SetUINT32(MFAttributesClsid.MF_MT_AUDIO_BLOCK_ALIGNMENT, 4);
Check = audioTypeIn.SetUINT32(MFAttributesClsid.MF_MT_AUDIO_AVG_BYTES_PER_SECOND, 44100 * 4);
Check = audioTypeIn.SetUINT32(MFAttributesClsid.MF_MT_AUDIO_BITS_PER_SAMPLE, 16);
Check = audioTypeIn.SetUINT32(MFAttributesClsid.MF_MT_ALL_SAMPLES_INDEPENDENT, 1);

// and output AAC to the MP4 file
Check = MFExtern.MFCreateMediaType(out audioTypeOut);
Check = audioTypeOut.SetGUID(MFAttributesClsid.MF_MT_MAJOR_TYPE, MFMediaType.Audio);
Check = audioTypeOut.SetGUID(MFAttributesClsid.MF_MT_SUBTYPE, MFMediaType.AAC);
Check = audioTypeOut.SetUINT32(MFAttributesClsid.MF_MT_AUDIO_NUM_CHANNELS, 2);
Check = audioTypeOut.SetUINT32(MFAttributesClsid.MF_MT_AUDIO_SAMPLES_PER_SECOND, 44100);
Check = audioTypeOut.SetUINT32(MFAttributesClsid.MF_MT_AUDIO_BITS_PER_SAMPLE, 16);
Check = audioTypeOut.SetUINT32(MFAttributesClsid.MF_MT_ALL_SAMPLES_INDEPENDENT, 0);

// pass it to the sinkWriter
Check = sinkWriter.AddStream(audioTypeOut, out audioStreamIndex);
Check = sinkWriter.SetInputMediaType(audioStreamIndex, audioTypeIn, null); // this call fails

Now, when I try to pass in AAC, the setup code works perfectly. Makes me think I am mis-configuring the PCM IMFMediaType, but I can't spot the error.

FYI stock AAC encoder's input must be MFAudioFormat_PCM. Your LPCM might need a conversion which is not available as automatic. — Roman R.
@RomanR. Thanks, I changed it to PCM. No change - the same error appears. (And I confirmed that my "PCM" in the C# api has the same value as MFAudioFormat_PCM in the official SDK) — LTR
OK, so next thing not looking good is your MF_MT_ALL_SAMPLES_INDEPENDENT on AAC type. Why don't you comment it out completely. — Roman R.

Roman R. Roman R. · Accepted Answer · 2017-01-04T08:19:10

Audio media types can be initialized like this:

// Input PCM from memory
Check = MFExtern.MFCreateMediaType(out audioTypeIn);
Check = audioTypeIn.SetGUID(MFAttributesClsid.MF_MT_MAJOR_TYPE, MFMediaType.Audio);
Check = audioTypeIn.SetGUID(MFAttributesClsid.MF_MT_SUBTYPE, MFMediaType.PCM);
Check = audioTypeIn.SetUINT32(MFAttributesClsid.MF_MT_AUDIO_NUM_CHANNELS, 2);
Check = audioTypeIn.SetUINT32(MFAttributesClsid.MF_MT_AUDIO_SAMPLES_PER_SECOND, 44100);
Check = audioTypeIn.SetUINT32(MFAttributesClsid.MF_MT_AUDIO_BITS_PER_SAMPLE, 16);

// and output AAC to the MP4 file
Check = MFExtern.MFCreateMediaType(out audioTypeOut);
Check = audioTypeOut.SetGUID(MFAttributesClsid.MF_MT_MAJOR_TYPE, MFMediaType.Audio);
Check = audioTypeOut.SetGUID(MFAttributesClsid.MF_MT_SUBTYPE, MFMediaType.AAC);
Check = audioTypeOut.SetUINT32(MFAttributesClsid.MF_MT_AUDIO_NUM_CHANNELS, 2);
Check = audioTypeOut.SetUINT32(MFAttributesClsid.MF_MT_AUDIO_SAMPLES_PER_SECOND, 44100);
Check = audioTypeOut.SetUINT32(MFAttributesClsid.MF_MT_AUDIO_BITS_PER_SAMPLE, 16);

Other attributes will be derived, and incorrect use of MF_MT_ALL_SAMPLES_INDEPENDENT in your original code snippet prevents from initialization of the codec.

Media Foundation MP4 Encoding: IMFSinkWriter doesn't accept PCM input

1 Answers