Capturing Video Data to Screen and File

0
votes

I am writing a simple as possible demo application which reads the incoming data from a WebCam and both displays it to the screen and (optionally at the toggle of a button) records the video stream to the disk. To do this I implemented a standard WMF pipeline architecture from the WebCam which acts as a media source to an EVR which acts as the media sink. In order to implement the file capture functionality, I modified a simple MFT (the FrameCounter MFT) to also write the incoming samples to a Sink Writer if the Sink Writer object is available. Thus all data travels from the source to the sink via the MFT and, if a Sink Writer is available to the MFT, it will also write the sample data to the Sink Writer was well.

The MFT does in place processing and releases the sample just before returning from its output processing loop. The write to the sink writer just uses the same input sample in its WriteSample() call before the release happens.

The problem is that this code works. I expected to have to make a copy of the sample in the MFT and give that to the Sink Writer so that it could release it. This raises the following questions..

  1. Does the Sink Writer make its own copy of the sample or add its own reference to the sample? Does it release the sample given to it? It is my understanding that the Sink Writer can queue these samples for some time (if it wishes) before writing them. What is the general treatment of samples in the Sink Writers WriteSample() call.

  2. Is it just a fluke that this is working now and am I creating problems for later or on slower machines? Is the technique described above valid? If not, what would be the recommended approach. I have experienced no odd run-time errors with this.

  3. I was under the impression that when capturing data from a web cam that the timestamp on the sample needed to be rebased before writing it out. In other words, the first sample written had to have a timestamp of 0 and every subsequent sample timestamp adjusted based off of that. This does not appear to be necessary in this case. Is this time base reset a requirement (was it ever)? I note that the sample code in the FileCapture demo takes care to do this. Have Microsoft improved the Sink Writer in recent versions to cope with this?

I am working on Windows 10 in C# using WMF.Net. I realize this may be a special case with the CLR and its garbage collection – however, I would appreciate any WMF insights (in any language) you may be able to offer. The Sink Writer output format is H.264 – if that matters.

1
ms-media-foundation

1 Answers

0
votes

1- The Sink Writer is not really documented about what it does of samples. For efficiency, i suppose it just "AddRef" on the sample. If you create a new sample after each video capture, you just have to release it after WriteSample. There will be no problem with the Sink Writer.

2- For slower machines, it works the same. Perhaps you will have to drop samples for efficiency, but it has nothing to do with releasing samples.

3- The WriteSample method from IMFSinkWriter documentation does not explicitly tells if sample time/duration are needed. So i would say, set them if necessary.

EDIT

My explanation above is about C++. C# COM interop is another story.

I read the article about interop com. It seems that C # developers do not set objects to nullptr after calling Marshal.ReleaseComObject, and reuse them. They receive an InvalidComObjectException. If you AddRef, Release ans set to NULL, like in C++, i think you will not have problems (the RCW is also doing reference counting).

For the case where a COM objet is rewritten in managed code :

MediaFoundation components are native code, as far as i know. I doubt they change them to managed code. And even if it's happen, you will just have to suppress Marshal.ReleaseComObject, or now use IsComObject.