Software by Default

Article purpose

This article illustrates a basic introductory level explanation of the steps required to interface with the Microsoft Kinect for Windows sensor using the Kinect for Windows Sdk implemented in a Windows Forms application.

Introduction

Most of the introduction level articles and code samples I’ve come across tend to be implemented as WPF or XNA based applications. Interfacing to the Microsoft Kinect for Windows sensor from Windows Forms based applications can be achieved fairly effortlessly. I find when wanting to create test applications a Windows Forms implementation can be developed quickly with relatively little hassle.

This article details creating a Windows Forms application which displays a live Video feed from the Kinect sensor’s Color camera whilst providing an option in specifying the resolution/frame rate.

Sample source code

This article is accompanied by sample source code in a Visual Studio project which is available for download here.

Sample Source Overview

The sample source code provided with this article consists of a Windows Form which host a PictureBox control used to display the incoming Video feed. It is possible that more than one Kinect sensor might be connected, the user can therefore select a sensor from a ComboBox. Possible resolutions/frame rates can also be configured via an additional ComboBox.

Available Kinect Sensors

The sample application implements the KinectSensor.KinectSensors property to retrieve a list of available Kinect sensor. The associated ComboBox will be populated with references to available sensors.

private void PopulateAvailableSensors()
{
    cmbSensors.Items.Clear();

  
    foreach (KinectSensor sensor in KinectSensor.KinectSensors)
    {
        cmbSensors.Items.Add(sensor.UniqueKinectId);
        cmbSensors.SelectedItem = sensor.UniqueKinectId;
    }
}

  

Initial setup: Form Constructor

In the Form constructor we define the available display modes by adding the relevant enumeration values to the associated ComboBox.

public  MainForm() 
{ 
   InitializeComponent(); 

 
   cmbDisplayMode.Items.Add(ColorImageFormat.RgbResolution640x480Fps30); 
   cmbDisplayMode.Items.Add(ColorImageFormat.RgbResolution1280x960Fps12); 
   cmbDisplayMode.SelectedIndex = 0; 

 
   PopulateAvailableSensors(); 
} 

In this scenario we specify:

• ColorImageFormat.RgbResolution640x480Fps30
• ColorImageFormat.RgbResolution1280x960Fps12

In the constructor the method PopulateAvailableSensors(), as described earlier, will be invoked after having specified available display modes.

Refreshing Available Sensors

Clicking the refresh button results in calling PopulateAvailableSensors(), which will re-determine available sensors and populate the associated ComboBox. Listed below is the click event handler subscribed to the Refresh button’s click event.

private void btnRefresh_Click(object sender, EventArgs e) 
{ 
    PopulateAvailableSensors(); 
} 

Activating the Sensor

When clicking on the “Activate Sensor” button the sample code first attempts to deactivate the sensor currently in use. After ensuring that resources related to the current sensor have been released the application attempts to assign an instance of the KinectSensor member object based on user selection.

private  void  btnActivateSensor_Click(object  sender, EventArgs  e) 
{ 
    if (cmbSensors.SelectedIndex != -1) 
    { 
        DeActivateSensor(); 

  
        string sensorID = cmbSensors.SelectedItem.ToString(); 

  
        foreach (KinectSensor sensor in KinectSensor.KinectSensors) 
        { 
           if (sensor.UniqueKinectId == sensorID) 
            { 
                kinectSensor = sensor; 
                SetupSensorVideoInput(); 
            } 
        } 
    } 
}

Sensor Video setup

In the sample application we implement the Kinect sensor’s ColorStream camera. The Kinect sensor exposes an event which informs subscribed event handlers when a video frame is available for processing. The following code snippet shows the code implemented in setting up the video format.

private void SetupSensorVideoInput() 
{ 
    if (kinectSensor != null) 
    { 
         imageFormat = (ColorImageFormat)cmbDisplayMode.SelectedItem; 
         kinectSensor.ColorStream.Enable(imageFormat);     

  
         kinectSensor.ColorFrameReady += 
         new EventHandler<ColorImageFrameReadyEventArgs>(kinectSensor_ColorFrameReady); 

         
         kinectSensor.Start(); 
     } 
}      

Processing Video Frames

The ColorFrameReady event informs subscribed event handlers whenever a Color frame becomes available for processing.

private void kinectSensor_ColorFrameReady(object  sender, ColorImageFrameReadyEventArgs  e) 
{ 
    ColorImageFrame colorFrame = e.OpenColorImageFrame();   

 
    if (colorFrame == null) 
    { 
         return; 
    }   

  
    Bitmap bitmapFrame = ColorImageFrameToBitmap(colorFrame); 
          
    picVideoDisplay.Image = bitmapFrame; 
}    

 
private static Bitmap ColorImageFrameToBitmap(ColorImageFrame colorFrame) 
{ 
    byte[] pixelBuffer = new  byte[colorFrame.PixelDataLength]; 
    colorFrame.CopyPixelDataTo(pixelBuffer);    

 
    Bitmap bitmapFrame = new Bitmap(colorFrame.Width, colorFrame.Height, 
        PixelFormat.Format32bppRgb); 

 
    BitmapData bitmapData = bitmapFrame.LockBits(new Rectangle(0, 0, 
                                     colorFrame.Width, colorFrame.Height), 
    ImageLockMode.WriteOnly, bitmapFrame.PixelFormat);   

  
    IntPtr intPointer = bitmapData.Scan0; 
    Marshal.Copy(pixelBuffer, 0, intPointer, colorFrame.PixelDataLength); 

           
    bitmapFrame.UnlockBits(bitmapData); 

  
    return  bitmapFrame; 
} 

The event handler parameters provides access to a ColorImageFrame object. Before the video frame can be displayed the ColorImageFrame object first needs to be converted to a memory Bitmap, which can then be displayed by the Form’s PictureBox. Notice the ColorImageFrameToBitmap method, it functions by extracting image data from the ColorImageFrame as a byte array, then creates a new memory Bitmap and copies the byte array of image data.

Deactivating a sensor

Earlier when we activated the sensor we deactivated the active sensor. Listed below is the relevant code snippet.

private void DeActivateSensor() 
{ 
    if (kinectSensor != null) 
    { 
        kinectSensor.Stop(); 


        kinectSensor.ColorFrameReady -= 
        new EventHandler<ColorImageFrameReadyEventArgs>
        (kinectSensor_ColorFrameReady); 

 
        kinectSensor.Dispose(); 
    } 
}