Preview issues for 1080P video using DirectShow

1
votes

I am using DirectShow in my application to capture video from webcams. I have issues while using cameras to preview and capture 1080P videos. Eg: HD Pro Webcam C910 camera of Logitech. 1080P video preview was very jerky and no HD clarity was observed. I could see that the enumerated device name was "USB Video Device"

Today we installed Logitech webcam software on these XP machines . In that application, we could see the 1080P video without any jerking. Also we recorded 1080P video in the Logitech application and saw them in high quality.

But when I test my application,

  1. I can see that the enumerated device name has been changed to "Logitech Pro Webcam C910" instead of the "USB Video Device" as in the previous case.

  2. The CPU eaten up by my application is 20%, but the process "SYSTEM" eats up 60%+ and the overall CPU revolves around 100%

    1. Even though the video quality has been greatly improved, the jerks are still there, may be due to the 100% CPU.

    2. When I closed my application, the high CPU utlizaton by "System" process goes away.

Regarding my application - It uses ICaptureGraphBuilder2::RenderStream to create Preview and Capture streams.

In Capture Stream, I connect Camera filter to NULL renderer with sample grabber as the intermediate filter.

In preview stream, I have 

g_pBuild->RenderStream(&PIN_CATEGORY_PREVIEW,&MEDIATYPE_Video,cam,NULL,NULL);

Preview is displayed on a windows as specified using IVideoWindow interface. I use the following

g_vidWin->put_Owner((OAHWND)(HWND)hWnd);
g_vidWin->put_WindowStyle(WS_CHILD | WS_CLIPSIBLINGS);
g_vidWin->put_MessageDrain((OAHWND)hWnd);

I tried setting Frame rate to different values ( AvgTimePerFrame = 500000 ( 20 fps ) and 666667(15 fps) etc.

But all the trials, still give the same result. Clarity has become more, but some jerks still remain and CPU is almost 100% due to 60+ % utlilization by "System". When I close my video application, usage by "System" goes back to 1-2 %.

Any help on this is most welcome.

Thanks in advance,

1
directshowpreviewframe-rate
Maybe a little more info would help. What's your CPU consumption by SYSTEM when you run the Logitech application? Maybe that's just the device driver using up your cycles. I'm not sure changing AvgTimePerFrame will affect the camera's actual frame rate. Where are you setting that? I don't have a C910, but I have a couple of other Logitech cameras, and their frame rates seem to be governed by the resolution and exposure settings. - Stevens Miller
When the Logitech application runs, SYSTEM doesn't use CPU more than 2% I was under the impression that changing AvgTimePerFrame will change the Frame Rate. Could you tell me the exact procedure in terms of DirectShow settings/interface methods, by which Frame Rate can be set? - Jo Bell
With my Logitechs, the only way I know to affect the frame rate is to change the resolution and exposure settings. The camera makes its own decision about what rate it's going to use. I'm not sure, but I think the average rate in the VIDEOINFO structure is something your source filter sets to advise other filters about the rate they should anticipate. It doesn't change the source filter's own rate; it helps other filters know what to expect. Your camera is going to pump out samples at whatever rate it wants. See msdn.microsoft.com/en-us/library/windows/desktop/… - Stevens Miller
Thanks Stevens. By the way, do you have any pointers on the excessive CPU usage by the "SYSTEM" process in Windows XP? Which forum will be the ideal place? Logitech forums? - Jo Bell
No, that's pretty puzzling. What else is going on in your application? Do you see this kind of CPU consumption if you build a graph with the filters you mentioned in graphedt.exe? - Stevens Miller

1 Answers

0
votes

Use IAMStreamConfig.SetFormat() to select the frame rate, dimensions, color space, and compression of the output streams (Capture and Preview) from a capture device.

Aside: The comment above of "It doesn't change the source filter's own rate" is completely wrong. The whole purpose of this interface is to define the output format and framerate of the captured video.

Use IAMStreamConfig.GetStreamCaps() to determine what frames rates, dimensions, color spaces, and compression formats are available. Most cameras provide a number different formats.

It sounds like the fundamental issue you're facing is that USB bandwidth (at least prior to USB3) can't sustain 30fps 1080P without compression. I'm most familiar with the Microsoft LifeCam Studio family of USB cameras, and these devices perform hardware compression to send the video over the wire, and then eat up a substantial fraction of your CPU on the receiving end converting the compressed video from Motion JPEG into a YUV format. Presumably the Logitech cameras work in a similar fashion.

The framerate that cameras produce is influenced by the additional workload of performing auto-focus, auto-color correction, and auto-exposure in software. Try disabling all these features on your camera if possible. In the era of Skype, camera software and hardware has become less attentive to maintaining a high framerate in favor of better image quality.

The DirectShow timing model for capture continues to work even if the camera can't produce frames at the requested rate as long as the camera indicates that frames are missing. It does this using "dropped frame" count field which rides along with each captured frame. The sum of the dropped frames plus the "real" frames must equal the requested frame rate set via IAMStreamConfig.SetFormat().

Using the LifeCam Studio on an I7 I have captured at 30fps 720p with preview, compressed to H.264 and written an .mp4 file to disk using around 30% of the CPU, but only if all the auto-focus/color/exposure settings on the camera are disabled.