Kinect Depth and IR image in UWP (remove color)

Question

I am working with Kinect V2 UWP C# for XBOX and windows. I followed Kinect UWP demo for this purpose. I was able to read and display frames as also shown in Camera Frame sample but I noticed that the Depth and IR images are in color for example: Kinect studio and UWP application output

I am new this and have tried to search but not found a clear answer. Can anyone help please? I would really appreciate this.

These are false colour images. The colour merely separates the distance between the readings (pixels) in both IR and depth. For eg. farther or unreadable objects are black whereas nearer objects go from gray to white. Same for IR where different values are represented by colour to distinguish them. — Atif Anwer
Is it possible to remove false colour from these images? In Kinect studio and on wpf application for windows this seems to be possible but I am finding it difficult to do this in UWP. — Peri
I have done it in custom WPF applications, however I cannot say about UWP apps. I would definitely be possible but youll have to dig a little deeper in the documentation i guess. Just read around the method that displays the frame in the window. There must be some argument that is being called that can be removed/changed and that is basically displaying false colours. In any case, it is just a visual representation and not changing the base data in any way. — Atif Anwer

Peri Peri · Accepted Answer · 2020-08-11T19:33:14

I tried many things and finally found a way to get images without pseudo colors.

Since XAML only displays in format Bgra8, it needed to be converted. It helped processing frames separately for color and depth as well.

I also needed to update my Windows 10 version to 10.0.19041.0 or later.

//clrFrame.
            var buffFrame = clrFrame?.BufferMediaFrame;
            
            // Get the Individual Frame
            var vidFrame = clrFrame?.VideoMediaFrame;
            {
                if (vidFrame == null) return;

                
                // create a UWP SoftwareBitmap and copy Frame into Bitmap
                SoftwareBitmap sbt = new SoftwareBitmap(vidFrame.SoftwareBitmap.BitmapPixelFormat, vidFrame.SoftwareBitmap.PixelWidth, vidFrame.SoftwareBitmap.PixelHeight);
                vidFrame.SoftwareBitmap.CopyTo(sbt);

                // PixelFormat needs to be in 8bit BGRA for Xaml writable bitmap
                if (sbt.BitmapPixelFormat != BitmapPixelFormat.Bgra8)
                    sbt = SoftwareBitmap.Convert(vidFrame.SoftwareBitmap, BitmapPixelFormat.Bgra8);
                
                if (source != null)
                {
                    // To write out to writable bitmap which will be used with ImageElement, it needs to run
                    // on UI Thread thus we use Dispatcher.RunAsync()...
                    var ignore = Dispatcher.RunAsync(Windows.UI.Core.CoreDispatcherPriority.Normal, () =>
                    {
                        // This code runs on UI Thread
                        // Create the writableBitmap for ImageElement display
                        extBitmap = new WriteableBitmap(sbt.PixelWidth, sbt.PixelHeight);

                        // Copy contents from UWP software Bitmap
                        // There are other ways of doing this instead of the double copy, 1st copy earlier
                        // this is a second copy.
                        sbt.CopyToBuffer(extBitmap.PixelBuffer);
                        extBitmap.Invalidate();

                        // Set the imageElement source
                        var ig = source.SetBitmapAsync(sbt);
                        imgView.Source = source;

                    });

                }
            }

The following project sample helps with this problem. I had to create processing for IR and depth and pass appropriate parameters.

https://github.com/dngoins/KinectUWPApps/tree/master/WorkingWithMediaCaptureFramesSolution

Kinect Depth and IR image in UWP (remove color)

1 Answers