Kinect Depth Image only partly visible

Question

I am new to Kinect and C#. I am trying to get the Depth Image from the Kinect, convert it to a bitmap to perform some OpenCV operations and then display it. The problem is, I am getting only a third of the depth image and the rest is completely black(as seen in the picture). This is not the raw depth image but the image that I receive after painting.

enter image description here

Here is the code-

image and image1 are the two image canvas i have for display.

void DepthFrameReady(object sender, DepthImageFrameReadyEventArgs e)
    {

        DepthImageFrame Image;
        Bitmap bm;
        using (Image = e.OpenDepthImageFrame())
        {


           if (Image != null)
            {
            this.shortpixeldata = new short[Image.PixelDataLength];
            this.depthFrame32 = new byte[Image.Width * Image.Height * Bgr32BytesPerPixel];


            bmp = new Bitmap(Image.Width, Image.Height, System.Drawing.Imaging.PixelFormat.Format32bppRgb);
            Image.CopyPixelDataTo(this.shortpixeldata);

            byte[] convertedDepthBits = this.ConvertDepthFrame(this.shortpixeldata, ((KinectSensor)sender).DepthStream);


            BitmapData bmapdata = bmp.LockBits(
                                new System.Drawing.Rectangle(0, 0, Image.Width, Image.Height),
                                ImageLockMode.WriteOnly,
                                bmp.PixelFormat);


            IntPtr ptr = bmapdata.Scan0;
            Marshal.Copy(convertedDepthBits, 0, ptr, Image.PixelDataLength);
            bmp.UnlockBits(bmapdata);

            MemoryStream ms1 = new MemoryStream(); 
            bmp.Save(ms1, System.Drawing.Imaging.ImageFormat.Jpeg);
            System.Windows.Media.Imaging.BitmapImage bImg = new System.Windows.Media.Imaging.BitmapImage();
            bImg.BeginInit();
            bImg.StreamSource = new MemoryStream(ms1.ToArray());
            bImg.EndInit();

            image.Source = bImg;

                if (bmp != null)
            {

                Image<Bgr, Byte> currentFrame = new Image<Bgr, Byte>(bmp); 

                Image<Gray, Byte> grayImage = currentFrame.Convert<Gray, Byte>().PyrDown().PyrUp();
                Image<Gray, Byte> Dest = new Image<Gray, Byte>(grayImage.Size);
                CvInvoke.cvCanny(grayImage, Dest, 10, 60, 3);


                image1.Source = ToBitmapSource(Dest);

                CalculateFps();
            }



            }



            else
            {
                System.Diagnostics.Debug.WriteLine("depth bitmap empty :/");
            }
        }
    }


        private byte[] ConvertDepthFrame(short[] depthFrame, DepthImageStream depthStream)
        {
            System.Diagnostics.Debug.WriteLine("depthframe len :{0}", depthFrame.Length);

        for (int i16 = 0, i32 = 0; i16 < depthFrame.Length && i32 < this.depthFrame32.Length; i16++, i32 += 4)
        {

        int realDepth = depthFrame[i16] >> DepthImageFrame.PlayerIndexBitmaskWidth;


        byte Distance = 0;


        int MinimumDistance = 800;
        int MaximumDistance = 4096;


        if (realDepth > MinimumDistance)
        {


        //White = Close
        //Black = Far
        Distance = (byte)(255-((realDepth-MinimumDistance)*255/(MaximumDistance-MinimumDistance)));


        this.depthFrame32[i32 + RedIndex] = (byte)(Distance);
        this.depthFrame32[i32 + GreenIndex] = (byte)(Distance);
        this.depthFrame32[i32 + BlueIndex] = (byte)(Distance);
        }

        else
        {
        this.depthFrame32[i32 + RedIndex] = 0;
        this.depthFrame32[i32 + GreenIndex] = 150;
        this.depthFrame32[i32 + BlueIndex] = 0;
        }
        }

        return this.depthFrame32;
        }

I tried different PixelFormats to no avail. I can't figure out the problem. Does someone have any idea what I'm doing wrong? Thanks

Nicholas Pappas Nicholas Pappas · Accepted Answer · 2013-03-15T16:17:04

I would suggest using a WritableBitmap in order to copy the depth image to a viewable format. In the case of the PixelFormat, that information is available in the depth image itself so you should be using the same format for the WritableBitmap as is being captured.

Have you looked at any of the examples provided by Microsoft? You can find them at the Kinect for Windows Samples CodePlex page. There are several samples that demonstrate how to copy the depth data into a WritableBitmap and then output it. For example, here is the DepthFrameReady callback function of the "DepthBasics-WPF" sample application:

/// <summary>
/// Event handler for Kinect sensor's DepthFrameReady event
/// </summary>
/// <param name="sender">object sending the event</param>
/// <param name="e">event arguments</param>
private void SensorDepthFrameReady(object sender, DepthImageFrameReadyEventArgs e)
{
    using (DepthImageFrame depthFrame = e.OpenDepthImageFrame())
    {
        if (depthFrame != null)
        {
            // Copy the pixel data from the image to a temporary array
            depthFrame.CopyDepthImagePixelDataTo(this.depthPixels);

            // Get the min and max reliable depth for the current frame
            int minDepth = depthFrame.MinDepth;
            int maxDepth = depthFrame.MaxDepth;

            // Convert the depth to RGB
            int colorPixelIndex = 0;
            for (int i = 0; i < this.depthPixels.Length; ++i)
            {
                // Get the depth for this pixel
                short depth = depthPixels[i].Depth;

                // To convert to a byte, we're discarding the most-significant
                // rather than least-significant bits.
                // We're preserving detail, although the intensity will "wrap."
                // Values outside the reliable depth range are mapped to 0 (black).

                // Note: Using conditionals in this loop could degrade performance.
                // Consider using a lookup table instead when writing production code.
                // See the KinectDepthViewer class used by the KinectExplorer sample
                // for a lookup table example.
                byte intensity = (byte)(depth >= minDepth && depth <= maxDepth ? depth : 0);

                // Write out blue byte
                this.colorPixels[colorPixelIndex++] = intensity;

                // Write out green byte
                this.colorPixels[colorPixelIndex++] = intensity;

                // Write out red byte                        
                this.colorPixels[colorPixelIndex++] = intensity;

                // We're outputting BGR, the last byte in the 32 bits is unused so skip it
                // If we were outputting BGRA, we would write alpha here.
                ++colorPixelIndex;
            }

            // Write the pixel data into our bitmap
            this.colorBitmap.WritePixels(
                new Int32Rect(0, 0, this.colorBitmap.PixelWidth, this.colorBitmap.PixelHeight),
                this.colorPixels,
                this.colorBitmap.PixelWidth * sizeof(int),
                0);
        }
    }
}

The complete code for this particular class can be found here: http://kinectforwindows.codeplex.com/SourceControl/changeset/view/861462899ae7#v1.x/ToolkitSamples1.6.0/C#/DepthBasics-WPF/MainWindow.xaml.cs

The "Kinect Explorer" example is another good one to review, as it examines all three streams at once. It requires library that is not included in the CodePlex repository, but can be found in the Kinect for Windows Toolkit.

Kinect Depth Image only partly visible

2 Answers