Kinect 2 green screen effect

Question

I am making a simple application in C# wpf met de Kinect 2. I want to create a green screen effect if users are detected. I am using the CoordinateMappingBasics example from the SDK. The problem is dat the edges of the users are "white" and not so nice looking. is there any way to achieve a blur effect around the edges? is there any way to apply a mask? i will appreciate any help!

// We'll access the body index data directly to avoid a copy
using (KinectBuffer bodyIndexData = bodyIndexFrame.LockImageBuffer())
{
    unsafe
    {
        byte* bodyIndexDataPointer = (byte*)bodyIndexData.UnderlyingBuffer;

        int colorMappedToDepthPointCount = this.colorMappedToDepthPoints.Length;

        fixed (DepthSpacePoint* colorMappedToDepthPointsPointer = this.colorMappedToDepthPoints)
        {
            // Treat the color data as 4-byte pixels
            uint* bitmapPixelsPointer = (uint*)this.bitmap.BackBuffer;

            // Loop over each row and column of the color image
            // Zero out any pixels that don't correspond to a body index
            for (int colorIndex = 0; colorIndex < colorMappedToDepthPointCount; ++colorIndex)
            {
                float colorMappedToDepthX = colorMappedToDepthPointsPointer[colorIndex].X;
                float colorMappedToDepthY = colorMappedToDepthPointsPointer[colorIndex].Y;

                // The sentinel value is -inf, -inf, meaning that no depth pixel corresponds to this color pixel.
                if (!float.IsNegativeInfinity(colorMappedToDepthX) &&
                    !float.IsNegativeInfinity(colorMappedToDepthY))
                {
                    // Make sure the depth pixel maps to a valid point in color space
                    int depthX = (int)(colorMappedToDepthX + 0.5f);
                    int depthY = (int)(colorMappedToDepthY + 0.5f);


                    // If the point is not valid, there is no body index there.
                    if ((depthX >= 0) && (depthX < depthWidth) && (depthY >= 0) && (depthY < depthHeight))
                    {
                        int depthIndex = (depthY * depthWidth) + depthX;


                        // If we are tracking a body for the current pixel, do not zero out the pixel
                        if (bodyIndexDataPointer[depthIndex] != 0xff)
                        {
                            continue;

                        }
                    }
                }

                bitmapPixelsPointer[colorIndex] = 0;
            }
        }

        this.bitmap.AddDirtyRect(new Int32Rect(0, 0, this.bitmap.PixelWidth, this.bitmap.PixelHeight));
    }
}
}

If you are facing similar problem like here, then you might be using a older version of Kinect v2 sdk. Upgrading your Kinect v2 sdk to latest version will remove the white color effect, according to the video in the link. — Rafaf Tahsin
I am using the last version of the SDK, the problem remains, but thanks for the suggestion. — Manuel

Jack.S Jack.S · Accepted Answer · 2016-03-02T00:00:05

This is kind of a hard thing to work around. My suggestion would be to look into a object detection script, matlab has a few examples. You could then combine the Kinects Depth data and the object detection box to more accurately round off the edges.

My advice with the Kinect is to always have a highly contrasting background to the user. Ensure you have good even lighting and try to maximize the distance between the person and any object/walls behind them.

Kinect 2 green screen effect

1 Answers