Rendering bitmap with XNA slow when grabbing image from webcam, why?

Question

I've been working for a few weeks on grabbing a webcam image and rendering it on a windows form, but have been struggeling with speed issues the entire time. I need at least a frame rate of 10 Hz to be able to update my background process.

I started of using a pictureBox but the solution I've ended up with is to create a XNA-panel inside my Form, and then rendering the image as a background-sprite by converting the bitmap into a Texture2D using a script I found here.

The problem I've encountered now and have not been able to solve is; when I load a Bitmap in the code by calling the bitmap constructor like below, everything runs smoothly and I can get high fps. This is what I did during testing and was very happy with the results.

Bitmap image = new Bitmap(320, 240);

But as soon as I send the bitmap that I grab from the webcam it takes a lot longer to render for some reason I can not fathom. To my knowledge of bitmaps the images are the same format, it's just the colour of the pixels that are different. What I checked for format is size (320*240), resolution (96) and pixel format (Format32bppArgb). Am I missing anything?

This is how I grab the image from the webcam:

VideoCaptureDevices = new FilterInfoCollection(FilterCategory.VideoInputDevice);

        FinalVideoSource = new VideoCaptureDevice(VideoCaptureDevices[0].MonikerString);
        FinalVideoSource.DesiredFrameSize = new Size(320, 240);
        FinalVideoSource.DesiredFrameRate = fps;
        FinalVideoSource.NewFrame += new NewFrameEventHandler(FinalVideoSource_NewFrame);

void FinalVideoSource_NewFrame(object sender, NewFrameEventArgs eventArgs)
    {
        // create bitmap from frame
        image = eventArgs.Frame.Clone(new Rectangle(0, 0, 320, 240), PixelFormat.Format32bppArgb);
...

This is my draw function in XNA:

protected override void Draw()
    {
        backTexture = GetTexture(GraphicsDevice, image);                

        GraphicsDevice.Clear(Microsoft.Xna.Framework.Color.CornflowerBlue);

        // TODO: Add your drawing code here
        sprites.Begin();
        Vector2 pos = new Vector2(0, 0);
        sprites.Draw(backTexture, pos, Microsoft.Xna.Framework.Color.White);
        sprites.End();
    }


private Texture2D GetTexture(GraphicsDevice dev, System.Drawing.Bitmap bmp)
    {
        int[] imgData = new int[bmp.Width * bmp.Height];
        Texture2D texture = new Texture2D(dev, bmp.Width, bmp.Height);

        unsafe
        {
            // lock bitmap
            System.Drawing.Imaging.BitmapData origdata =
                bmp.LockBits(new System.Drawing.Rectangle(0, 0, bmp.Width, bmp.Height), System.Drawing.Imaging.ImageLockMode.ReadOnly, bmp.PixelFormat);

            uint* byteData = (uint*)origdata.Scan0;

            // Switch bgra -> rgba
            for (int i = 0; i < imgData.Length; i++)
            {
                byteData[i] = (byteData[i] & 0x000000ff) << 16 | (byteData[i] & 0x0000FF00) | (byteData[i] & 0x00FF0000) >> 16 | (byteData[i] & 0xFF000000);
            }

            // copy data
            System.Runtime.InteropServices.Marshal.Copy(origdata.Scan0, imgData, 0, bmp.Width * bmp.Height);

            byteData = null;

            // unlock bitmap
            bmp.UnlockBits(origdata);
        }

        texture.SetData(imgData);

        return texture;
    }

I would be very grateful if someone could help me with this since I'm stuck now. The community here has been great and I managed to get this far without asking before, which is amazing since I have no prior experience with C# or XNA. With that in mind I realize I might be missing something simple or just approaching this the wrong way.

I've narrowed it down to the bitmap image loading. The only thing I change when using the newly constructed bitmap is to simply overwrite the one from the webcam before processing it in XNA.

My question right now is really, am I missing something with how the bitmaps are constructed which can explain the big difference in rendering speed? Is the conversion to Texture2D the problem here? But I don't see how a different image could affect the speed of that conversion.

EdgarT EdgarT · Accepted Answer · 2012-08-31T05:35:29

Ok. I don't know what is the problem exactly. But I can give you some comments.
-First, set the frame rate of the XNA game equal or less that your webcam fps. By default XNA run at 60fps, so you are calling the GetTexture() method twice for each frame of your webcam if you are using 30fps. In the Initialize code:

TargetElapsedTime = TimeSpan.FromSeconds(1f/webcam fps)

If that don't work... you can try this code to convert from bitmap to texture.

protected override void Draw()
{
    //Unset the texture from the GraphicsDevice
    for (int i = 0; i < 16; i++)
            {
                if (Game.GraphicsDevice.Textures[i] == backTexture)
                {
                    Game.GraphicsDevice.Textures[i] = null;
                    break;
                }
            }

    backTexture.SetData<byte>(image.GetBytes());                

    GraphicsDevice.Clear(Microsoft.Xna.Framework.Color.CornflowerBlue);

    // TODO: Add your drawing code here
    sprites.Begin();
    Vector2 pos = new Vector2(0, 0);
    sprites.Draw(backTexture, pos, Microsoft.Xna.Framework.Color.White);
    sprites.End();

}


public static byte[] GetBytes(this Bitmap bitmap)
{
    var data = bitmap.LockBits(new System.Drawing.Rectangle(0, 0, bitmap.Width, bitmap.Height), 
        System.Drawing.Imaging.ImageLockMode.ReadOnly, bitmap.PixelFormat);

    // calculate the byte size: for PixelFormat.Format32bppArgb (standard for GDI bitmaps) it's the hight * stride
    int bufferSize = data.Height * data.Stride; // stride already incorporates 4 bytes per pixel

    // create buffer
    byte[] bytes = new byte[bufferSize];

    // copy bitmap data into buffer
    Marshal.Copy(data.Scan0, bytes, 0, bytes.Length);

    // unlock the bitmap data
    bitmap.UnlockBits(data);

    return bytes;

}

I was testing this code and works fine. Hope this help.

Rendering bitmap with XNA slow when grabbing image from webcam, why?

1 Answers