Adding Sound and Input Sound on Media Foundation Sink Writer

Question

I'm able to produce mp4 file with my screen recording app but it only captures screen and has no sound. Now I want it to also make the recording output an audio. What I want is to encode the audio that is coming from the pc which is the direct sound and blend it with the sound that coming from the mic. I'm using sharpdx and mediafoundation.net Below is my code how I initialize the audiodevice in sharpdx

        var audioDevices = SharpDX.DirectSound.DirectSoundCapture.GetDevices();
        SharpDX.DirectSound.DirectSound dxSoundCapture = new SharpDX.DirectSound.DirectSound(audioDevices.First().DriverGuid);

and below is how I initialize the mediatype for the audio in mediafoundation

    IMFMediaType audioTypeIn = null;
IMFMediaType audioTypeOut = null;

// Create the Audio input type 
if (Succeeded(hr)) hr = (int)MFExtern.MFCreateMediaType(out audioTypeOut);
if (Succeeded(hr)) hr = (int)audioTypeOut.SetGUID(MFAttributesClsid.MF_MT_MAJOR_TYPE, MFMediaType.Audio);
if (Succeeded(hr)) hr = (int)audioTypeOut.SetGUID(MFAttributesClsid.MF_MT_SUBTYPE, MFMediaType.AAC);
if (Succeeded(hr)) hr = (int)audioTypeOut.SetUINT32(MFAttributesClsid.MF_MT_AUDIO_NUM_CHANNELS, 2);
if (Succeeded(hr)) hr = (int)audioTypeOut.SetUINT32(MFAttributesClsid.MF_MT_AUDIO_SAMPLES_PER_SECOND, 44100);
if (Succeeded(hr)) hr = (int)audioTypeOut.SetUINT32(MFAttributesClsid.MF_MT_AUDIO_BITS_PER_SAMPLE, 16);
if (Succeeded(hr)) hr = (int)audioTypeOut.SetUINT32(MFAttributesClsid.MF_MT_ALL_SAMPLES_INDEPENDENT, 1);
if (Succeeded(hr)) hr = (int)sinkWriter.AddStream(audioTypeOut, out streamAudIndex);

// Create the Audio input type 
if (Succeeded(hr)) hr = (int)MFExtern.MFCreateMediaType(out audioTypeIn);
if (Succeeded(hr)) hr = (int)audioTypeIn.SetGUID(MFAttributesClsid.MF_MT_MAJOR_TYPE, MFMediaType.Audio);
if (Succeeded(hr)) hr = (int)audioTypeIn.SetGUID(MFAttributesClsid.MF_MT_SUBTYPE, MFMediaType.PCM);
if (Succeeded(hr)) hr = (int)audioTypeIn.SetUINT32(MFAttributesClsid.MF_MT_AUDIO_NUM_CHANNELS, 2);
if (Succeeded(hr)) hr = (int)audioTypeIn.SetUINT32(MFAttributesClsid.MF_MT_AUDIO_SAMPLES_PER_SECOND, 44100);
if (Succeeded(hr)) hr = (int)audioTypeIn.SetUINT32(MFAttributesClsid.MF_MT_AUDIO_BITS_PER_SAMPLE, 16);
if (Succeeded(hr)) hr = (int)audioTypeIn.SetUINT32(MFAttributesClsid.MF_MT_ALL_SAMPLES_INDEPENDENT, 1);
if (Succeeded(hr)) hr = (int)sinkWriter.SetInputMediaType(streamAudIndex, audioTypeIn, null);

My question is how can I supply the audio sample to my sinkwriter? In my video I basically supply the texture2d coming from SharpDX and make a sample buffer out of it.

Roman R. Roman R. · Accepted Answer · 2017-06-16T06:03:44

You are preparing Sink Writer to accept a PCM audio stream of the following configuration: 44100 samples/sec, stereo, 16 bits/sample.

You are supposed to either obtain media samples with respective data, or create them yourself using MFCreateSample and friends. See also Working with Media Samples on MSDN.

Once you have a media sample, IMFSinkWriter::WriteSample sends it to encoding.

Adding Sound and Input Sound on Media Foundation Sink Writer

1 Answers