Morphology Filters | Software by Default

Archive for the 'Morphology Filters' Category

C# How to: Fuzzy Blur Filter

Article Purpose

This article serves to illustrate the concepts involved in implementing a Fuzzy Blur Filter. This filter results in rendering non-photo realistic images which express a certain artistic effect.

Frog: Filter Size 19×19

Sample Source Code

This article is accompanied by a sample source code Visual Studio project which is available for download here.

Using the Sample Application

The sample source code accompanying this article includes a Windows Forms based test application. The concepts explored throughout this article can be replicated/tested using the sample application.

When executing the sample application the user interface exposes a number of configurable options:

Loading and Saving Images – Users are able to load source/input images from the local file system by clicking the Load Image button. Clicking the Save Image button allow users to save filter result images.
Filter Size – The specified filter size affects the filter intensity. Smaller filter sizes result in less blurry images being rendered, whereas larger filter sizes result in more blurry images being rendered.
Edge Factors – The contrast of fuzzy image noise expressed in resulting images depend on the specified edge factor values. Values less than one result in detected image edges being darkened and values greater than one result in detected image edges being lightened.

The following image is a screenshot of the Fuzzy Blur Filter sample application in action:

Frog: Filter Size 9×9

Fuzzy Blur Overview

The Fuzzy Blur Filter relies on the interference of image noise when performing edge detection in order to create a fuzzy effect. In addition image blurring results from performing a mean filter.

The steps involved in performing a Fuzzy Blur Filter can be described as follows:

Edge Detection and Enhancement – Using the first edge factor specified enhance image edges by performing Boolean Edge detection. Being sensitive to image noise, a fair amount of detected image edges will actually be image noise in addition to actual image edges.
Mean Filter Blur – Using the edge enhanced image created in the previous step perform a mean filter blur. The enhanced edges will be blurred since a Mean filter does not have edge preservation properties. The size of the Mean filter implemented depends on a user specified value.
Edge Detection and Enhancement – Using the Mean filter blurred image created in the previous step once again perform Boolean Edge detection, enhancing detected edges according to the second edge factor specified.

Frog: Filter Size 9×9

Mean Filter Blurring

A Mean Filter Blur, also known as a Box Blur, can be performed through image convolution. The size of the matrix/kernel implemented when preforming image convolution will be determined through user input.

Every matrix/kernel element should be set to one. The resulting value should be multiplied by a factor value equating to one divided by the matrix/kernel size. As an example, a matrix/kernel size of 3×3 can be expressed as follows:

An alternative expression can also be:

Frog: Filter Size 9×9

Boolean Edge Detection without a local threshold

When performing Boolean Edge Detection a local threshold should be implemented in order to exclude image noise. In this article we rely on the interference of image noise in order to render a fuzzy image effect. By not implementing a local threshold when performing Boolean Edge detection the sample source code ensures sufficient interference from image noise.

The steps involved in performing Boolean Edge Detection without a local threshold can be described as follows:

Calculate Neighbourhood Mean – Iterate each pixel forming part of the source/input image. Using a 3×3 matrix size calculate the mean value of the neighbourhood surrounding the pixel currently being iterated.
Create Mean comparison Matrix – Once again using a 3×3 matrix size compare each neighbourhood pixel to the newly calculated mean value. Create a temporary 3×3 size matrix, each matrix element’s value should be the result of mean comparison. Should the value expressed by a neighbourhood pixel exceed the mean value the corresponding temporary matrix element should be set to one. When the calculated mean value exceeds the value of a neighbourhood pixel the corresponding temporary matrix element should be set to zero.
Compare Edge Masks – Using sixteen predefined edge masks compare the temporary matrix created in the previous step to each edge mask. If the temporary matrix matches one of the predefined edge masks multiply the specified factor to the pixel currently being iterated.

Note: A detailed article on Boolean Edge detection implementing a local threshold can be found here: C# How to: Boolean Edge Detection

Frog: Filter Size 9×9

The sixteen predefined edge masks each represent an image edge in a different direction. The predefined edge masks can be expressed as:

Frog: Filter Size 13×13

Implementing a Mean Filter

The sample source code defines the MeanFilter method, an extension method targeting the Bitmap class. The definition listed as follows:

private static Bitmap MeanFilter(this Bitmap sourceBitmap, 
                                 int meanSize)
{
    byte[] pixelBuffer = sourceBitmap.GetByteArray(); 
    byte[] resultBuffer = new byte[pixelBuffer.Length];


    double blue = 0.0, green = 0.0, red = 0.0;
    double factor = 1.0 / (meanSize * meanSize);


    int imageStride = sourceBitmap.Width * 4;
    int filterOffset = meanSize / 2; 
    int calcOffset = 0, filterY = 0, filterX = 0; 


    for (int k = 0; k + 4 < pixelBuffer.Length; k += 4)
    {
        blue = 0; green = 0; red = 0;
        filterY = -filterOffset;
        filterX = -filterOffset;


        while (filterY <= filterOffset)
        {
            calcOffset = k + (filterX * 4) +
            (filterY * imageStride); 


            calcOffset = (calcOffset < 0 ? 0 :
            (calcOffset >= pixelBuffer.Length - 2 ?
            pixelBuffer.Length - 3 : calcOffset));


            blue += pixelBuffer[calcOffset];
            green += pixelBuffer[calcOffset + 1];
            red += pixelBuffer[calcOffset + 2];


            filterX++;


            if (filterX > filterOffset)
            {
                filterX = -filterOffset;
                filterY++;
            }
        }


        resultBuffer[k] = ClipByte(factor * blue);
        resultBuffer[k + 1] = ClipByte(factor * green);
        resultBuffer[k + 2] = ClipByte(factor * red);
        resultBuffer[k + 3] = 255;
    }


    return resultBuffer.GetImage(sourceBitmap.Width, sourceBitmap.Height);
}

Frog: Filter Size 19×19

Implementing Boolean Edge Detection

Boolean Edge detection is performed in the sample source code through the implementation of the BooleanEdgeDetectionFilter method. This method has been defined as an extension method targeting the Bitmap class.

The following code snippet provides the definition of the BooleanEdgeDetectionFilter extension method:

public static Bitmap BooleanEdgeDetectionFilter( 
       this Bitmap sourceBitmap, float edgeFactor) 
{
    byte[] pixelBuffer = sourceBitmap.GetByteArray(); 
    byte[] resultBuffer = new byte[pixelBuffer.Length]; 
    Buffer.BlockCopy(pixelBuffer, 0, resultBuffer, 
                     0, pixelBuffer.Length); 


    List<string> edgeMasks = GetBooleanEdgeMasks(); 
 
    int imageStride = sourceBitmap.Width * 4; 
    int matrixMean = 0, pixelTotal = 0; 
    int filterY = 0, filterX = 0, calcOffset = 0; 
    string matrixPatern = String.Empty; 


    for (int k = 0; k + 4 < pixelBuffer.Length; k += 4) 
    {
        matrixPatern = String.Empty; 
        matrixMean = 0; pixelTotal = 0; 
        filterY = -1; filterX = -1; 


        while (filterY < 2) 
        {
            calcOffset = k + (filterX * 4) + 
            (filterY * imageStride); 


            calcOffset = (calcOffset < 0 ? 0 : 
            (calcOffset >= pixelBuffer.Length - 2 ? 
            pixelBuffer.Length - 3 : calcOffset)); 
 
            matrixMean += pixelBuffer[calcOffset]; 
            matrixMean += pixelBuffer[calcOffset + 1]; 
            matrixMean += pixelBuffer[calcOffset + 2]; 


            filterX += 1; 


            if (filterX > 1) 
             { filterX = -1; filterY += 1; } 
        }


        matrixMean = matrixMean / 9; 
        filterY = -1; filterX = -1; 


        while (filterY < 2) 
        {
            calcOffset = k + (filterX * 4) + 
            (filterY * imageStride); 


            calcOffset = (calcOffset < 0 ? 0 : 
            (calcOffset >= pixelBuffer.Length - 2 ? 
            pixelBuffer.Length - 3 : calcOffset)); 


            pixelTotal = pixelBuffer[calcOffset]; 
            pixelTotal += pixelBuffer[calcOffset + 1]; 
            pixelTotal += pixelBuffer[calcOffset + 2]; 
 
            matrixPatern += (pixelTotal > matrixMean 
                                       ? "1" : "0"); 
            filterX += 1; 


            if (filterX > 1) 
            { filterX = -1; filterY += 1; } 
        } 


        if (edgeMasks.Contains(matrixPatern)) 
        {
            resultBuffer[k] = 
            ClipByte(resultBuffer[k] * edgeFactor); 


            resultBuffer[k + 1] = 
            ClipByte(resultBuffer[k + 1] * edgeFactor); 


            resultBuffer[k + 2] = 
            ClipByte(resultBuffer[k + 2] * edgeFactor); 
        }
    }


    return resultBuffer.GetImage(sourceBitmap.Width, sourceBitmap.Height); 
}

Frog: Filter Size 13×13

The predefined edge masks implemented in mean comparison have been wrapped by the GetBooleanEdgeMasks method. The definition as follows:

public static List<string> GetBooleanEdgeMasks() 
{
    List<string> edgeMasks = new List<string>(); 


    edgeMasks.Add("011011011"); 
    edgeMasks.Add("000111111"); 
    edgeMasks.Add("110110110"); 
    edgeMasks.Add("111111000"); 
    edgeMasks.Add("011011001"); 
    edgeMasks.Add("100110110"); 
    edgeMasks.Add("111011000"); 
    edgeMasks.Add("111110000"); 
    edgeMasks.Add("111011001"); 
    edgeMasks.Add("100110111"); 
    edgeMasks.Add("001011111"); 
    edgeMasks.Add("111110100"); 
    edgeMasks.Add("000011111"); 
    edgeMasks.Add("000110111"); 
    edgeMasks.Add("001011011"); 
    edgeMasks.Add("110110100"); 


    return edgeMasks; 
}

Frog: Filter Size 19×19

Implementing a Fuzzy Blur Filter

The FuzzyEdgeBlurFilter method serves as the implementation of a Fuzzy Blur Filter. As discussed earlier a Fuzzy Blur Filter involves enhancing image edges through Boolean Edge detection, performing a Mean Filter blur and then once again performing Boolean Edge detection. This method has been defined as an extension method targeting the Bitmap class.

The following code snippet provides the definition of the FuzzyEdgeBlurFilter method:

public static Bitmap FuzzyEdgeBlurFilter(this Bitmap sourceBitmap,  
                                         int filterSize,  
                                         float edgeFactor1,  
                                         float edgeFactor2) 
{
    return  
    sourceBitmap.BooleanEdgeDetectionFilter(edgeFactor1). 
    MeanFilter(filterSize).BooleanEdgeDetectionFilter(edgeFactor2); 
}

Frog: Filter Size 3×3

Sample Images

This article features a number of sample images. All featured images have been licensed allowing for reproduction. The following images feature as sample images:

Tyler’s Tree Frog
Attribution: LiquidGhoul. This file has been released into the public domain by its author, LiquidGhoul. This applies worldwide.
Download from Wikipedia.

Phyllobates Terribilis
Attribution: Wilfried Berns. This file is licensed under the Creative Commons Attribution-Share Alike 2.0 Germany license.
Download from Wikipedia.

Dendropsophus Microcephalus
Attribution: Brian Gratwicke. This file is licensed under the Creative Commons Attribution 2.0 Generic license.
Download from Wikipedia

Panamanian Golden Frog
Attribution: Brian Gratwicke. This file is licensed under the Creative Commons Attribution 2.0 Generic license.
Download from Wikipedia.

C# How to: Image Boundary Extraction

Published July 21, 2013 Augmented Reality , Blogging , C# , Code Samples , Edge Detection , Extension Methods , Graphic Filters , Graphics , How to , Image Arithmetic , Image Filters , Image Processing , Image Transform , Learn Everyday , Microsoft , Morphology Filters , New Version , Opensource , Tip , Update 5 Comments
Tags: Augmented Reality, Binary Image, Bitmap ARGB, Bitmap Filters, Bitmap.LockBits, BitmapData, Boundary Sharpen, Boundary Tracing, C#, Code Sample, Computer vision, Converting Images, Edge Detect, Edge Detection, Edge enhance, Edge Tracing, Feature extraction, Graphic Filters, Image, Image Boundary Extraction, Image Difference, Image Dilation, Image Erosion, Image Intensity, Image Morphology, Image processing, Image Sharpen, Image Transform, Machine vision, Morphologial Filters, Morphological Edge Detection, Morphological Transform, Pixel Filters, Structured Element

Article Purpose

This article explores various image processing concepts, which feature in combination when implementing Image Boundary Extraction. Concepts covered within this article include: Morphological Image Erosion and Image Dilation, Image Addition and Subtraction, Boundary Sharpening, Boundary Tracing and Boundary Extraction.

Parrot: Boundary Extraction, 3×3, Red, Green, Blue

Sample Source Code

This article is accompanied by a sample source code Visual Studio project which is available for download here.

Using the Sample Application

This article’s accompanying sample source code includes the definition of a sample application. The sample application serves as an implementation of the concepts discussed in this article. In using the sample application concepts can be easily tested and replicated.

The sample application has been defined as a Windows Forms application. The user interface enables the user to configure several options which influence the output produced from image filtering processes. The following section describes the options available to a user when executing the sample application:

Loading and Saving files – Users can specify source/input images through clicking the Load Image button. If desired, resulting filtered images can be saved to the local file system when clicking the Save Image button.
Filter Type – The types of filters implemented represent variations on Image Boundary Extraction. The supported filter types are: Conventional Boundary extraction, Boundary Sharpening and Boundary Tracing.
Filter Size – Filter intensity/strength will mostly be reliant on the filter size implemented. A Filter size represents the number of neighbouring pixels examined when applying filters.
Colours Applied – The sample source code and sample application provides functionality allowing a filter to only effect user specified colour components. Colour components are represented in the form of an RGB colour scheme. The inclusion or exclusion of the colour components Red, Green and Blue will be determined through user configuration.
Structuring Element – As mentioned, the Filter Size option determines the size of neighbourhood pixels examined. The Structuring Element’s setup determine the neighbouring pixels within the pixel neighbourhood size bounds that should be used as input when calculating filter results.

The following image is a screenshot of the Image Boundary Extraction sample application in action:

Parrot: Boundary Extraction, 3×3, Green

Morphological Boundary Extraction

Image Boundary Extraction can be considered a method of Image Edge Detection. In contrast to more commonly implemented gradient based edge detection methods, Image Boundary Extraction originates from Morphological Image Filters.

When drawing a comparison, Image Boundary Extraction and Morphological Edge Detection express strong similarities. Morphological Edge Detection results from the difference in image erosion and image dilation. Considered from a different point of view, creating one image expressing thicker edges and another image expressing thinner edges provides the means to calculate the difference in edges.

Image Boundary Extraction implements the same concept as Morphological Edge Detection. The base concept can be regarded as calculating the difference between two images which rendered the same image, but expressing a difference in image edges. Image Boundary Extraction relies on calculating the difference between either image erosion and the source image or image dilation and the source image. The difference between image erosion and image dilation in most cases result in more of difference than the difference between image erosion and the source image or image dilation and the source image. The result of Image Boundary Extraction representing less of a difference than Morphological Edge Detection can be observed in Image Boundary Extraction being expressed in finer/smaller width lines.

Difference of Gaussians is another method of edge detection which functions along the same basis. Edges are determined by calculating the difference between two images, each having been filtered from the same source image, using a Gaussian blur of differing intensity levels.

Parrot: Boundary Extraction, 3×3, Red, Green, Blue

Boundary Sharpening

The concept of Boundary Sharpening refers to enhancing or sharpening the boundaries or edges expressed in a source/input image. Boundaries can be easily determined or extracted as discussed earlier when exploring Boundary Extraction.

The steps involved in performing Boundary Sharpening can be described as follows:

Extract Boundaries – Determine boundaries by performing image dilation and calculating the difference between the dilated image and the source image.
Match Source Edges and Extracted Boundaries – The boundaries extracted in the previous step represent the difference between image dilation and the original source image. Ensure that extracted boundaries match the source image through performing image dilation on a copy of the source/input image.
Emphasise Extracted boundaries in source image – Perform image addition using the extracted boundaries image and dilated copy of the source image.

Parrot: Boundary Extraction, 3×3, Red, Green, Blue

Boundary Tracing

Boundary Tracing refers to applying image filters which result in image edges/boundaries appearing darker or more pronounced. This type of filter also relies on Boundary Extraction.

Boundary Tracing can be implemented in two steps, described as follows:

Extract Boundaries – Determine boundaries by performing image dilation and calculating the difference between the dilated image and the source image.
Emphasise Extracted boundaries in source image – Subtract the extracted boundaries from the original source image.

Parrot: Boundary Extraction, 3×3, Red, Green, Blue

Implementing Morphological Erosion and Dilation

The accompanying sample source code defines the MorphologyOperation method, defined as an extension method targeting the Bitmap class. In terms of parameters this method expects a two dimensional array representing a structuring element. The other required parameter represents an enumeration value indicating which Morphological Operation to perform, either erosion or dilation.

The following code snippet provides the definition in full:

private static Bitmap MorphologyOperation(this Bitmap sourceBitmap,
                                          bool[,] se,
                                          MorphologyOperationType morphType,
                                          bool applyBlue = true,
                                          bool applyGreen = true,
                                          bool applyRed = true)
{ 
    BitmapData sourceData =
               sourceBitmap.LockBits(new Rectangle(0, 0,
               sourceBitmap.Width, sourceBitmap.Height),
               ImageLockMode.ReadOnly,
               PixelFormat.Format32bppArgb);


    byte[] pixelBuffer = new byte[sourceData.Stride *
                                  sourceData.Height];


    byte[] resultBuffer = new byte[sourceData.Stride *
                                   sourceData.Height];


    Marshal.Copy(sourceData.Scan0, pixelBuffer, 0,
                               pixelBuffer.Length);


    sourceBitmap.UnlockBits(sourceData);


    int filterOffset = (se.GetLength(0) - 1) / 2;
    int calcOffset = 0, byteOffset = 0;
    byte blueErode = 0, greenErode = 0, redErode = 0;
    byte blueDilate = 0, greenDilate = 0, redDilate = 0;


    for (int offsetY = 0; offsetY <
        sourceBitmap.Height - filterOffset; offsetY++)
    {
        for (int offsetX = 0; offsetX <
            sourceBitmap.Width - filterOffset; offsetX++)
        {
            byteOffset = offsetY * sourceData.Stride +
                         offsetX * 4;


            blueErode = 255; greenErode = 255; redErode = 255;
            blueDilate = 0; greenDilate = 0; redDilate = 0;


            for (int filterY = -filterOffset;
                    filterY <= filterOffset; filterY++)
            {
                for (int filterX = -filterOffset;
                    filterX <= filterOffset; filterX++)
                {
                    if (se[filterY + filterOffset,  
                           filterX + filterOffset] == true)
                    {
                        calcOffset = byteOffset +
                                     (filterX * 4) +
                        (filterY * sourceData.Stride);


                        calcOffset = (calcOffset < 0 ? 0 :
                        (calcOffset >= pixelBuffer.Length + 2 ?
                        pixelBuffer.Length - 3 : calcOffset));


                        blueDilate =  
                        (pixelBuffer[calcOffset] > blueDilate ?
                        pixelBuffer[calcOffset] : blueDilate);


                        greenDilate =  
                        (pixelBuffer[calcOffset + 1] > greenDilate ?
                        pixelBuffer[calcOffset + 1] : greenDilate);


                        redDilate =  
                        (pixelBuffer[calcOffset + 2] > redDilate ?
                        pixelBuffer[calcOffset + 2] : redDilate);


                        blueErode =  
                        (pixelBuffer[calcOffset] < blueErode ?
                        pixelBuffer[calcOffset] : blueErode);


                        greenErode =  
                        (pixelBuffer[calcOffset + 1] < greenErode ?
                        pixelBuffer[calcOffset + 1] : greenErode);


                        redErode =  
                        (pixelBuffer[calcOffset + 2] < redErode ?
                        pixelBuffer[calcOffset + 2] : redErode);
                    }
                }
            }


            blueErode = (applyBlue ? blueErode : pixelBuffer[byteOffset]);
            blueDilate = (applyBlue ? blueDilate : pixelBuffer[byteOffset]);


            greenErode = (applyGreen ? greenErode : pixelBuffer[byteOffset + 1]);
            greenDilate = (applyGreen ? greenDilate : pixelBuffer[byteOffset + 1]);


            redErode = (applyRed ? redErode : pixelBuffer[byteOffset + 2]);
            redDilate = (applyRed ? redDilate : pixelBuffer[byteOffset + 2]);


            if (morphType == MorphologyOperationType.Erosion)
            {
                resultBuffer[byteOffset] = blueErode;
                resultBuffer[byteOffset + 1] = greenErode;
                resultBuffer[byteOffset + 2] = redErode;
            }
            else if (morphType == MorphologyOperationType.Dilation)
            {
                resultBuffer[byteOffset] = blueDilate;
                resultBuffer[byteOffset + 1] = greenDilate;
                resultBuffer[byteOffset + 2] = redDilate;
            }


            resultBuffer[byteOffset + 3] = 255;
        }
    }


    Bitmap resultBitmap = new Bitmap(sourceBitmap.Width, sourceBitmap.Height);
    BitmapData resultData = resultBitmap.LockBits(new Rectangle(0, 0,
                            resultBitmap.Width, resultBitmap.Height),
                            ImageLockMode.WriteOnly,
                            PixelFormat.Format32bppArgb);


    Marshal.Copy(resultBuffer, 0, resultData.Scan0,
                               resultBuffer.Length);


    resultBitmap.UnlockBits(resultData);


    return resultBitmap;
}

Parrot: Boundary Extraction, 3×3, Red, Green

Implementing Image Addition

The sample source code encapsulates the process of combining two separate images through means of addition. The AddImage method serves as a single declaration of image addition functionality. This method has been defined as an extension method targeting the Bitmap class. Boundary Sharpen filtering implements image addition.

The following code snippet provides the definition of the AddImage extension method:

private static Bitmap AddImage(this Bitmapsource Bitmap, 
                               Bitmap addBitmap)
{
    BitmapData sourceData =
               sourceBitmap.LockBits(new Rectangle (0, 0,
               sourceBitmap.Width, sourceBitmap.Height),
               ImageLockMode.ReadOnly,
               PixelFormat.Format32bppArgb);


    byte[] resultBuffer = new byte[sourceData.Stride *
                                  sourceData.Height];


    Marshal.Copy(sourceData.Scan0, resultBuffer, 0,
                               resultBuffer.Length);


    sourceBitmap.UnlockBits(sourceData);


    BitmapData addData =
               addBitmap.LockBits(new Rectangle(0, 0,
               addBitmap.Width, addBitmap.Height),
               ImageLockMode.ReadOnly,
               PixelFormat.Format32bppArgb);


    byte[] addBuffer = new byte[addData.Stride *
                                  addData.Height];


    Marshal.Copy(addData.Scan0, addBuffer, 0,
                               addBuffer.Length);


    addBitmap.UnlockBits(addData);


    for (int k = 0; k + 4 < resultBuffer.Length &&
                    k + 4 < addBuffer.Length; k += 4)
    {
        resultBuffer[k] =
        AddColors(resultBuffer[k], addBuffer[k]);
 
        resultBuffer[k + 1] =
        AddColors(resultBuffer[k + 1], addBuffer[k + 1]);
 
        resultBuffer[k + 2] =
        AddColors(resultBuffer[k + 2], addBuffer[k + 2]);
 
        resultBuffer[k + 3] = 255;
    }


    Bitmap resultBitmap = new Bitmap(sourceBitmap.Width,
                                     sourceBitmap.Height);


    BitmapData resultData =
               resultBitmap.LockBits(new Rectangle(0, 0,
               resultBitmap.Width, resultBitmap.Height),
               ImageLockMode.WriteOnly,
               PixelFormat.Format32bppArgb);


    Marshal.Copy(resultBuffer, 0, resultData.Scan0,
                               resultBuffer.Length);


    resultBitmap.UnlockBits(resultData);


    return resultBitmap;
}

private static byte AddColors(byte color1, byte color2) 
{
    int result = color1 + color2; 

 
    return (byte)(result < 0 ? 0 : (result > 255 ? 255 : result)); 
}

Parrot: Boundary Extraction, 3×3, Red, Green, Blue

Implementing Image Subtraction

In a similar fashion regarding the AddImage method the sample code defines the SubractImage method. By definition this method serves as an extension method targeting the Bitmap class. Image subtraction has been implemented in Boundary Extraction and Boundary Tracing.

The definition of the SubtractImage method listed as follows:

private static Bitmap SubtractImage(this Bitmap sourceBitmap,  
                                         Bitmap subtractBitmap) 
{
    BitmapData sourceData = 
               sourceBitmap.LockBits(new Rectangle(0, 0, 
               sourceBitmap.Width, sourceBitmap.Height), 
               ImageLockMode.ReadOnly, 
               PixelFormat.Format32bppArgb); 


    byte[] resultBuffer = new byte[sourceData.Stride * 
                                  sourceData.Height]; 


    Marshal.Copy(sourceData.Scan0, resultBuffer, 0, 
                               resultBuffer.Length); 


    sourceBitmap.UnlockBits(sourceData); 


    BitmapData subtractData = 
               subtractBitmap.LockBits(new Rectangle(0, 0, 
               subtractBitmap.Width, subtractBitmap.Height), 
               ImageLockMode.ReadOnly, 
               PixelFormat.Format32bppArgb); 


    byte[] subtractBuffer = new byte[subtractData.Stride * 
                                  subtractData.Height]; 


    Marshal.Copy(subtractData.Scan0, subtractBuffer, 0, 
                               subtractBuffer.Length); 


    subtractBitmap.UnlockBits(subtractData); 


    for (int  k = 0; k + 4 < resultBuffer.Length &&  
                    k + 4 < subtractBuffer.Length; k += 4) 
    { 
        resultBuffer[k] =  
        SubtractColors(resultBuffer[k], subtractBuffer[k]); 


        resultBuffer[k + 1] =  
        SubtractColors(resultBuffer[k + 1], subtractBuffer[k + 1]); 


        resultBuffer[k + 2] = 
        SubtractColors(resultBuffer[k + 2], subtractBuffer[k + 2]); 


        resultBuffer[k + 3] = 255; 
    }


    Bitmap resultBitmap = new Bitmap (sourceBitmap.Width, 
                                     sourceBitmap.Height); 


    BitmapData resultData = 
               resultBitmap.LockBits(new Rectangle (0, 0, 
               resultBitmap.Width, resultBitmap.Height), 
               ImageLockMode.WriteOnly, 
               PixelFormat.Format32bppArgb); 


    Marshal.Copy(resultBuffer, 0, resultData.Scan0, 
                               resultBuffer.Length); 


    resultBitmap.UnlockBits(resultData); 


    return resultBitmap; 
}

private static byte SubtractColors(byte color1, byte color2) 
{
    int result = (int)color1 - (int)color2; 

 
    return (byte)(result < 0 ? 0 : result); 
}

Parrot: Boundary Extraction, 3×3, Green

Implementing Image Boundary Extraction

In the sample source code processing Image Boundary Extraction can be achieved when invoking the BoundaryExtraction method. Defined as an extension method, the BoundaryExtraction method targets the Bitmap class.

As discussed earlier, this method performs Boundary Extraction through subtracting the source image from a dilated copy of the source image.

The following code snippet details the definition of the BoundaryExtraction method:

private static Bitmap
BoundaryExtraction(this Bitmap sourceBitmap, 
                   bool[,] se, bool applyBlue = true, 
                   bool applyGreen = true, bool applyRed = true) 
{
    Bitmap resultBitmap = 
           sourceBitmap.MorphologyOperation(se,  
           MorphologyOperationType.Dilation, applyBlue,  
                                  applyGreen, applyRed); 

 
    resultBitmap = resultBitmap.SubtractImage(sourceBitmap); 

  
    return resultBitmap; 
}

Parrot: Boundary Extraction, 3×3, Red, Blue

Implementing Image Boundary Sharpening

Boundary Sharpening in the sample source code has been implemented through the definition of the BoundarySharpen method. The BoundarySharpen extension method targets the Bitmap class. The following code snippet provides the definition:

private static Bitmap 
BoundarySharpen(this Bitmap sourceBitmap, 
                bool[,] se, bool applyBlue = true, 
                bool applyGreen = true, bool applyRed = true) 
{
    Bitmap resultBitmap = 
           sourceBitmap.BoundaryExtraction(se, applyBlue, 
                                           applyGreen, applyRed); 


    resultBitmap = sourceBitmap.MorphologyOperation(se,  
                   MorphologyOperationType.Dilation, applyBlue,  
                   applyGreen, applyRed).AddImage(resultBitmap); 


    return resultBitmap; 
}

Parrot: Boundary Extraction, 3×3, Green

Implementing Image Boundary Tracing

Boundary Tracing has been defined through the BoundaryTrace extension method, which targets the Bitmap class. Similar to the BoundarySharpen method this method performs Boundary Extraction, the result of which serves to be subtracted from the original source image. Subtracting image boundaries/edges result in those boundaries/edges being darkened, or traced. The definition of the BoundaryTracing extension method detailed as follows:

private static Bitmap
BoundaryTrace(this Bitmap sourceBitmap, 
              bool[,] se, bool applyBlue = true, 
              bool applyGreen = true, bool applyRed = true) 
{
    Bitmap resultBitmap =
    sourceBitmap.BoundaryExtraction(se, applyBlue,  
                                    applyGreen, applyRed); 


    resultBitmap = sourceBitmap.SubtractImage(resultBitmap); 

 
    return resultBitmap; 
}

Parrot: Boundary Extraction, 3×3, Green, Blue

Implementing a Wrapper Method

The BoundaryExtractionFilter method is the only method defined as publicly accessible. Following convention, this method’s definition signals the method as an extension method targeting the Bitmap class. This method has the intention of acting as a wrapper method, a single method capable of performing Boundary Extraction, Boundary Sharpening and Boundary Tracing, depending on method parameters.

The definition of the BoundaryExtractionFilter method detailed by the following code snippet:

public static Bitmap
BoundaryExtractionFilter(this Bitmap sourceBitmap, 
                         bool[,] se, BoundaryExtractionFilterType  
                         filterType, bool applyBlue = true, 
                         bool applyGreen = true, bool applyRed = true) 
{
    Bitmap resultBitmap = null; 

 
    if (filterType == BoundaryExtractionFilterType.BoundaryExtraction) 
    {
        resultBitmap =  
        sourceBitmap.BoundaryExtraction(se, applyBlue, 
                                        applyGreen, applyRed); 
    }
    else if (filterType == BoundaryExtractionFilterType.BoundarySharpen) 
    {
        resultBitmap =  
        sourceBitmap.BoundarySharpen(se, applyBlue,  
                                     applyGreen, applyRed); 
    }
    else if (filterType == BoundaryExtractionFilterType.BoundaryTrace) 
    {
        resultBitmap =  
        sourceBitmap.BoundaryTrace(se, applyBlue,  
                                   applyGreen, applyRed); 
    }

 
    return resultBitmap; 
}

Parrot: Boundary Extraction, 3×3, Red, Green, Blue

Sample Images

This article features a number of sample images. All featured images have been licensed allowing for reproduction. The following images feature as sample images:

Scarlet Macaw at Diergaarde Blijdorp, Rotterdam, Netherlands.
Attributed to: Jar0d. This file is licensed under the Creative Commons Attribution-Share Alike 2.0 Generic license.
Download from Wikipedia

A Scarlet Macaw flying away from the photographer.
Attributed to: Robert01. This file is licensed under the Creative Commons Attribution-Share Alike 3.0 Germany license.
Download from Wikipedia

Blue-and-yellow Macaw in flight.
Attributed to: Luc Viatour – www.lucnix.be. This file is licensed under the Creative Commons Attribution 2.0 Generic license.
Download from Wikipedia

Two Green-winged Macaws (also known as the Red-and-green Macaw) resting at Seaport Village, San Diego, USA.
Attributed to: Dave Fayram. This file is licensed under the Creative Commons Attribution 2.0 Generic license.
Download from Wikimedia.org

A Scarlet Macaw riding a small tricycle at an exhibition in Spain.
Attributed to: The Torch. This file is licensed under the Creative Commons Attribution 2.0 Generic license.
Download from Wikipedia.

Long-tailed Broadbill (Psarisomus dalhousiae), Kaeng Krachan National Park, Phetchaburi, Thailand.
Attributed to User-JJ Harrison. This file is licensed under the Creative Commons Attribution-Share Alike 3.0 Unported license.
Download from Wikipedia.

C# How to: Image Cartoon Effect

Published June 2, 2013 Augmented Reality , C# , Code Samples , Edge Detection , Extension Methods , Graphic Filters , Graphics , How to , Image Arithmetic , Image Convolution , Image Filters , Image Processing , Microsoft , Morphology Filters , New Version , Opensource , Update 22 Comments
Tags: Animation Effect, Augmented Reality, Binary Image, Bitmap ARGB, Bitmap Filters, Bitmap.LockBits, BitmapData, Boolean Edge Detection, C#, Cartoon Effect, Change detection, Code Sample, Computer vision, Converting Images, Convolution filters, Convolution Kernel, Convolution matrix, Edge Masks, Feature extraction, Gradient Edge Detection, Graphic Filters, Image, Image Edge Detection, Image Gradient, Image Morphology, Image processing, Image Sharpen, Image Sharpness, Image Transform, Line Detection, Machine vision, Matrix, Pixel Filters

Article purpose

In this article we explore the tasks related to creating a Cartoon Effect from images which reflect real world non-animated scenarios. When applying a Cartoon Effect it becomes possible with relative ease to create images appearing to have originated from a drawing/animation.

Cartoon version of Steve Ballmer: Low Pass 3×3, Threshold 65.

Sample source code

This article is accompanied by a sample source code Visual Studio project which is available for download here.

CPU: Gaussian 7×7, Threshold 84

Using the Sample Application

A Sample Application has been included as part of the sample source code accompanying this article. The Sample Application is a Windows Forms based application which enables a user to specify source/input images, apply various methods of implementing the Cartoon Effect. In addition users are able to save generated images to the local file system.

When using the Sample Application click the Load Image button to load image files from the local file system. On the right-hand side of the Sample application’s user interface, users are provided with two configuration options: Smoothing Filter and Threshold.

Rose: Gaussian 3×3 Threshold 28.

In this article and sample source code image detail and definition can be reduced through means of image smoothing filters. Several image smoothing options are available to the user, the following section serves as a discussion of each option.

None – When specifying the Smoothing Filter option None, no image smoothing operations will be performed on source/input images.

Gaussian 3×3 – Gaussian blur filters can be very effective at removing image noise, smoothing an image background, whilst still preserving the edges expressed in the sample/input image. A Convolution matrix/kernel of 3×3 dimensions result in slight image blurring.

Gaussian 5×5 – A Gaussian blur operation being implemented by making use of a convolution matrix/kernel defined with dimensions of 5×5. A slightly larger kernel results in an increased level of image blurring being expressed by output images. A greater level of image blurring equates to a larger degree of image noise reduction/removal.

Rose: Gaussian 7×7 Threshold 48.

Gaussian 7×7 – As can be expected when specifying a convolution matrix/kernel conforming to 7×7 size dimension an even more intense level of image blurring can be detected when looking at result images. Notice how increased levels of image blurring negatively affects the process of edge detection. Consider the following: In a scenario where too many elements are being detected as part of an edge as a result of image noise, specifying a higher level of image blurring should reduce edges being detected. The reasoning can be explained in terms of image blurring reducing image definition/detail, higher levels of image blurring will thus result in a greater level of image detail/definition reduction. Lower definition images are less likely to express the same level of detected edges when compared to higher definition images.

CPU: Median 3×3, Threshold 96.

Median 3×3 – When applying a Median filter to an image the resulting image should express a lesser degree of image noise. In other words, the Median filter can be considered as well suited to performing noise reduction. Also note that a Median filter under certain conditions has the ability to preserve the edges contained in an image. In the following section we explore the importance of edge detection in achieving a Cartoon Effect. Important concepts to take note of: The Median Filter when implemented on an image performs noise reduction whilst preserving image edges. In relation, edge detection represents a core concept/task when creating a Cartoon Effect. The Median Filter’s edge preservation property compliments the process of edge detection. When an image contains a low level of image noise the Median 3×3 Filter could be considered.

Median 5×5 – The 5×5 dimension implementation of the Median Filter result in producing images which exhibit a higher degree of smoothing and a lesser expression of image noise. If the 3×3 Median Filter fails to provide adequate levels of image smoothing and noise reduction the 5×5 Median Filter could be implemented.

Cartoon version of Steve Ballmer: Sharpen 3×3, Threshold 80.

Median 7×7 – The last Median Filter implemented by the sample source code conforms to a 7×7 size dimension. This filter variation results in a high level of image noise reduction. The trade off to more effective noise reduction will be expressed in result images appearing extremely smooth, in some scenarios perhaps overly so.

Mean 3×3 – The Mean Filter provides a different implementation towards achieving image smoothing and noise reduction.

Mean 5×5 – The 5×5 dimension Mean Filter variation serves as a more intense version of the Mean 3×3 Filter. Depending on the level of image noise and type of image noise a Mean Filter could prove a more efficient implementation in comparison to a Median Filter.

Low Pass 3×3 – In much the same fashion as Median and Mean Filters, a Low Pass Filter achieves images smoothing and noise reduction. Notice when comparing Median, Mean and Low Pass Filtering, the differences observed in output results are only expressed as slight differences. The most effective filter to apply should be seen as as being dependent on the input/source image characteristics.

CPU: Gaussian 3×3, Threshold 92.

Low Pass 5×5 – This filter variation being of a larger dimension serves as a more intense implementation of the Low Pass 3×3 Filter.

Sharpen 3×3 – In certain scenarios input/source images may already be smoothed/blurred to such an extent where the edge detection process performs below expectation. Edge detection can be improved when applying a image Sharpen filter.

Threshold values specified by the user through the user interface serves the purpose of enabling the user to finely control the extent/intensity of edges being detected. Implementing a higher Threshold value will have the result of less edges being detected. In order to reduce the level of image noise being detected as false edges the Threshold value should be increased. When too few edges are being detected the Threshold value should be decreased.

The following image is a screenshot of the Image Cartoon Effect Sample Application in action:

Explanation of the Cartoon Effect

The Cartoon Effect can be characterised as an image filter producing result images which appear similar to input/source images with the exception of having an animated appearance.

The Cartoon Effect consists of reducing image detail/definition whilst at the same instance performing edge detection. The resulting smoothed image and the edges detected in the source/input image should be combined, where detected edges are being expressed in the colour black. The final image reflects an appearance similar to that of an animated/artist drawn image.

Various methods of reducing image detail/definition are supported in the sample source code. Most methods consist of implementing image smoothing. The following configurable methods are implemented:

None
Gaussian 3×3
Gaussian 5×5
Gaussian 7×7
Median 3×3
Median 5×5
Median 7×7
Median 9×9
Mean 3×3
Mean 5×5
Low Pass 3×3
Low Pass 5×5
Sharpen 3×3

Rose: Mean 5×5 Threshold 37.

All of the filter methods listed above are implemented by means of Image Convolution. The size dimensions listed for each filter option relates to the dimension of the kernel/matrix being implemented by a filter.

When applying a filter, the intensity/extent will be determined by the size dimensions of the Convolution kernel/matrix implemented. Smaller kernel/matrix dimensions result in a filter being applied to a lesser extent. Larger kernel/matrix dimensions will result in the filter effect being more evident, being applied to a greater extent. Image noise reduction will be achieved when implementing a filter.

The Sample Source code implements Gradient Based Edge Detection using the original source/input image, therefore not being influenced by any smoothing operations. I have published an in-depth article on the topic of Gradient Based Edge Detection which can be located here: C# How to: Gradient Based Edge Detection.

Rose: Median 3×3 Threshold 37.

The Sample source code implements Gradient Based Edge Detection by means of iterating each pixel that forms part of the sample/input image. Whilst iterating image pixels the sample code calculate various gradients from the current pixel’s neighbouring pixels, on a per colour component basis (Red, Green and Blue). Referring to neighbouring pixels, calculations include the value of each of the surrounding pixels in regards to the pixel currently being iterated. Neighbouring pixel calculations are better know as kernel/window/matrix operations.

Note: Do not confuse image convolution and the method in which we iterate and calculate gradients. Although both methods have various aspects in common, image convolution is regarded as linear filter processing, whereas our method qualifies as a non-linear filter.

We calculate various gradients, which is to be compared against the user specified global threshold value. If a calculated gradient value exceeds the value of the user specified threshold the pixel currently being iterated will be considered as part of an edge.

The first gradients to be calculated involves the pixel directly above, below, left and right of the current pixel. A gradient will be calculated for each colour component. The gradient values being calculated can be considered as an indicator reflecting the rate of change. If the sum total of the calculated gradients exceed that of the global threshold the pixel will be considered as forming part an edge.

When the comparison of the threshold value and the total gradient value reflects in favour of the threshold the following set of gradients will be calculated. This process of calculating gradients will continue either until a gradient value exceeds the threshold or all gradients have been calculated.

If a pixel was detected as forming part of an edge, the pixel’s colour will be set to black. In the case of non-edge pixels, the original colour components from the source/input image will be used in setting the current pixel’s value.

Rose: Gaussian 3×3 Threshold 28

Implementing Cartoon Effects

The sample source code implementation can be divided into five distinct components: Cartoon Effect Filter, smoothing helper method, Median Filter implementation, Convolution implementation and the collection of pre-defined matrix/kernel values.

The sample source code defines the MedianFilter extension method targeting the Bitmap class. The following code snippet provides the definition:

 public static Bitmap MedianFilter(this Bitmap sourceBitmap, 
                                   int matrixSize) 
{ 
     BitmapData sourceData = 
                sourceBitmap.LockBits(new Rectangle(0, 0, 
                sourceBitmap.Width, sourceBitmap.Height), 
                ImageLockMode.ReadOnly, 
                PixelFormat.Format32bppArgb); 

   
     byte[] pixelBuffer = new byte[sourceData.Stride * 
                                   sourceData.Height]; 

   
     byte[] resultBuffer = new byte[sourceData.Stride * 
                                    sourceData.Height]; 

   
     Marshal.Copy(sourceData.Scan0, pixelBuffer, 0, 
                                pixelBuffer.Length); 

   
     sourceBitmap.UnlockBits(sourceData); 

   
     int filterOffset = (matrixSize - 1) / 2; 
     int calcOffset = 0; 

   
     int byteOffset = 0; 

   
     List<int> neighbourPixels = new List<int>(); 
     byte[] middlePixel; 

   
     for (int offsetY = filterOffset; offsetY < 
         sourceBitmap.Height - filterOffset; offsetY++) 
     { 
         for (int offsetX = filterOffset; offsetX < 
             sourceBitmap.Width - filterOffset; offsetX++) 
         { 
             byteOffset = offsetY * 
                          sourceData.Stride + 
                          offsetX * 4; 

   
             neighbourPixels.Clear(); 

   
             for (int filterY = -filterOffset; 
                 filterY <= filterOffset; filterY++) 
             {
                 for (int filterX = -filterOffset; 
                     filterX <= filterOffset; filterX++) 
                 {

   
                     calcOffset = byteOffset + 
                                  (filterX * 4) + 
                                  (filterY * sourceData.Stride); 

   
                     neighbourPixels.Add(BitConverter.ToInt32( 
                                      pixelBuffer, calcOffset)); 
                 } 
             } 

   
             neighbourPixels.Sort(); 
  
             middlePixel = BitConverter.GetBytes( 
                                neighbourPixels[filterOffset]); 

   
             resultBuffer[byteOffset] = middlePixel[0]; 
             resultBuffer[byteOffset + 1] = middlePixel[1]; 
             resultBuffer[byteOffset + 2] = middlePixel[2]; 
             resultBuffer[byteOffset + 3] = middlePixel[3]; 
         } 
     }

   
     Bitmap resultBitmap = new Bitmap (sourceBitmap.Width, 
                                      sourceBitmap.Height); 

   
     BitmapData resultData = 
                resultBitmap.LockBits(new Rectangle  (0, 0, 
                resultBitmap.Width, resultBitmap.Height), 
                ImageLockMode.WriteOnly, 
                PixelFormat.Format32bppArgb); 

   
     Marshal.Copy(resultBuffer, 0, resultData.Scan0, 
                                resultBuffer.Length); 

   
     resultBitmap.UnlockBits(resultData); 

   
     return resultBitmap; 
}

The SmoothingFilterType enumeration, defined by the sample source code, serves as a strongly typed definition of a the collection of implemented smoothing filters. The definition as follows:

 public enum SmoothingFilterType  
 {
     None, 
     Gaussian3x3, 
     Gaussian5x5, 
     Gaussian7x7, 
     Median3x3, 
     Median5x5, 
     Median7x7, 
     Median9x9, 
     Mean3x3, 
     Mean5x5, 
     LowPass3x3, 
     LowPass5x5, 
     Sharpen3x3, 
 }

The Matrix class contains the definition of all the two dimensional kernel/matrix values implemented when performing image convolution. The definition as follows:

public static class Matrix 
{ 
    public static double[,] Gaussian3x3 
    { 
        get 
        {
            return new double[,]   
             { { 1, 2, 1, },  
               { 2, 4, 2, },  
               { 1, 2, 1, }, }; 
        } 
    }
 
    public static double[,] Gaussian5x5 
    {
        get 
        { 
            return new double[,]   
             { { 2, 04, 05, 04, 2  },  
               { 4, 09, 12, 09, 4  },  
               { 5, 12, 15, 12, 5  }, 
               { 4, 09, 12, 09, 4  }, 
               { 2, 04, 05, 04, 2  }, }; 
        } 
    } 
 
    public static double[,] Gaussian7x7 
    {
        get 
        { 
            return new double[,]   
             { { 1,  1,  2,  2,  2,  1,  1, },  
               { 1,  2,  2,  4,  2,  2,  1, },  
               { 2,  2,  4,  8,  4,  2,  2, },  
               { 2,  4,  8, 16,  8,  4,  2, },  
               { 2,  2,  4,  8,  4,  2,  2, },  
               { 1,  2,  2,  4,  2,  2,  1, },  
               { 1,  1,  2,  2,  2,  1,  1, }, }; 
        } 
    } 
 
    public static double[,] Mean3x3 
    { 
        get 
        { 
            return new double[,]   
             { { 1, 1, 1, },  
               { 1, 1, 1, },  
               { 1, 1, 1, }, }; 
        } 
    } 
 
    public static double[,] Mean5x5 
    { 
        get 
        { 
            return new double[,]   
             { { 1, 1, 1, 1, 1, },  
               { 1, 1, 1, 1, 1, },  
               { 1, 1, 1, 1, 1, },  
               { 1, 1, 1, 1, 1, },  
               { 1, 1, 1, 1, 1, }, }; 
        } 
    } 
 
    public static double [,] LowPass3x3 
    { 
        get 
        { 
            return new double [,]   
             { { 1, 2, 1, },  
               { 2, 4, 2, },   
               { 1, 2, 1, }, }; 
        }
    } 
 
    public static double[,] LowPass5x5 
    { 
        get 
        { 
            return new double[,]   
             { { 1, 1,  1, 1, 1, },  
               { 1, 4,  4, 4, 1, },  
               { 1, 4, 12, 4, 1, },  
               { 1, 4,  4, 4, 1, },  
               { 1, 1,  1, 1, 1, }, }; 
        }
    }
 
    public static double[,] Sharpen3x3 
    { 
        get 
         {
            return new double[,]   
             { { -1, -2, -1, },  
               {  2,  4,  2, },   
               {  1,  2,  1, }, }; 
         }
    } 
}

Rose: Low Pass 3×3 Threshold 61

The SmoothingFilter extension method targets the Bitmap class. This method implements image convolution. The primary task performed by the SmoothingFilter extension method involves translating convolution filter options into the correct method calls. The definition as follows:

public static Bitmap SmoothingFilter(this Bitmap sourceBitmap, 
                            SmoothingFilterType smoothFilter = 
                            SmoothingFilterType.None) 
 {
    Bitmap inputBitmap = null; 

   
    switch (smoothFilter) 
    { 
        case SmoothingFilterType.None: 
            {
                inputBitmap = sourceBitmap; 
            } break; 
        case SmoothingFilterType.Gaussian3x3: 
            {
                inputBitmap = sourceBitmap.ConvolutionFilter( 
                           Matrix.Gaussian3x3, 1.0 / 16.0, 0); 
            } break; 
        case SmoothingFilterType.Gaussian5x5: 
            { 
                inputBitmap = sourceBitmap.ConvolutionFilter( 
                          Matrix.Gaussian5x5, 1.0 / 159.0, 0); 
            } break; 
        case SmoothingFilterType.Gaussian7x7: 
            { 
                inputBitmap = sourceBitmap.ConvolutionFilter( 
                          Matrix.Gaussian7x7, 1.0 / 136.0, 0); 
            } break; 
        case SmoothingFilterType.Median3x3: 
            { 
                inputBitmap = sourceBitmap.MedianFilter(3); 
            } break; 
        case SmoothingFilterType.Median5x5: 
            { 
                inputBitmap = sourceBitmap.MedianFilter(5); 
            } break; 
        case SmoothingFilterType.Median7x7: 
            { 
                inputBitmap = sourceBitmap.MedianFilter(7); 
            } break; 
        case SmoothingFilterType.Median9x9: 
            { 
                inputBitmap = sourceBitmap.MedianFilter(9); 
            } break; 
        case SmoothingFilterType.Mean3x3: 
            { 
                inputBitmap = sourceBitmap.ConvolutionFilter( 
                              Matrix.Mean3x3, 1.0 / 9.0, 0); 
            } break; 
        case SmoothingFilterType.Mean5x5: 
            { 
                inputBitmap = sourceBitmap.ConvolutionFilter( 
                              Matrix.Mean5x5, 1.0 / 25.0, 0); 
            } break; 
        case SmoothingFilterType.LowPass3x3: 
            { 
                inputBitmap = sourceBitmap.ConvolutionFilter( 
                              Matrix.LowPass3x3, 1.0 / 16.0, 0); 
            } break; 
        case SmoothingFilterType.LowPass5x5: 
            { 
                inputBitmap = sourceBitmap.ConvolutionFilter( 
                              Matrix.LowPass5x5, 1.0 / 60.0, 0); 
            } break; 
        case SmoothingFilterType.Sharpen3x3: 
            { 
                inputBitmap = sourceBitmap.ConvolutionFilter( 
                              Matrix.Sharpen3x3, 1.0 / 8.0, 0); 
            } break; 
    } 

   
    return inputBitmap; 
}

The ConvolutionFilter extension method which targets the Bitmap class implements image convolution. The definition as follows:

private static Bitmap ConvolutionFilter(this Bitmap sourceBitmap, 
                                          double[,] filterMatrix, 
                                               double factor = 1, 
                                                    int bias = 0) 
{ 
    BitmapData sourceData = sourceBitmap.LockBits(new Rectangle(0, 0, 
                             sourceBitmap.Width, sourceBitmap.Height), 
                                               ImageLockMode.ReadOnly, 
                                         PixelFormat.Format32bppArgb); 

   
    byte[] pixelBuffer = new byte[sourceData.Stride * sourceData.Height]; 
    byte[] resultBuffer = new byte[sourceData.Stride * sourceData.Height]; 

   
    Marshal.Copy(sourceData.Scan0, pixelBuffer, 0, pixelBuffer.Length); 
    sourceBitmap.UnlockBits(sourceData); 

   
    double blue = 0.0; 
    double green = 0.0; 
    double red = 0.0; 

   
    int filterWidth = filterMatrix.GetLength(1); 
    int filterHeight = filterMatrix.GetLength(0); 

   
    int filterOffset = (filterWidth - 1) / 2; 
    int calcOffset = 0; 

   
    int byteOffset = 0; 

   
    for (int offsetY = filterOffset; offsetY < 
        sourceBitmap.Height - filterOffset; offsetY++) 
     { 
        for (int offsetX = filterOffset; offsetX < 
            sourceBitmap.Width - filterOffset; offsetX++) 
         { 
            blue = 0; 
            green = 0; 
            red = 0; 

   
            byteOffset = offsetY * 
                         sourceData.Stride + 
                         offsetX * 4; 

   
            for (int filterY = -filterOffset; 
                filterY <= filterOffset; filterY++) 
             { 
                for (int filterX = -filterOffset; 
                    filterX <= filterOffset; filterX++) 
                 { 

   
                    calcOffset = byteOffset + 
                                 (filterX * 4) + 
                                 (filterY * sourceData.Stride); 

   
                    blue += (double)(pixelBuffer[calcOffset]) * 
                            filterMatrix[filterY + filterOffset, 
                                                filterX + filterOffset]; 

   
                    green += (double)(pixelBuffer[calcOffset + 1]) * 
                             filterMatrix[filterY + filterOffset, 
                                                filterX + filterOffset]; 

   
                    red += (double)(pixelBuffer[calcOffset + 2]) * 
                           filterMatrix[filterY + filterOffset, 
                                              filterX + filterOffset]; 
                 } 
             }

   
            blue = factor * blue + bias; 
            green = factor * green + bias; 
            red = factor * red + bias; 

   
            blue = (blue > 255 ? 255 : 
                   (blue < 0 ? 0 : 
                    blue)); 

   
            green = (green > 255 ? 255 : 
                    (green < 0 ? 0 : 
                     green)); 

   
            red = (red > 255 ? 255 : 
                  (red < 0 ? 0 : 
                   red));  

   
            resultBuffer[byteOffset] = (byte)(blue); 
            resultBuffer[byteOffset + 1] = (byte)(green); 
            resultBuffer[byteOffset + 2] = (byte)(red); 
            resultBuffer[byteOffset + 3] = 255; 
         } 
     }

   
    Bitmap resultBitmap = new Bitmap(sourceBitmap.Width, sourceBitmap.Height); 

   
    BitmapData resultData = resultBitmap.LockBits(new Rectangle (0, 0, 
                             resultBitmap.Width, resultBitmap.Height), 
                                              ImageLockMode .WriteOnly, 
                                         PixelFormat .Format32bppArgb); 

   
    Marshal.Copy(resultBuffer, 0, resultData.Scan0, resultBuffer.Length); 
    resultBitmap.UnlockBits(resultData); 

   
    return resultBitmap; 
}

Cartoon version of Steve Ballmer: Sharpen 3×3 Threshold 80

The CartoonEffectFilter extension method targets the Bitmap class. This method defines all the tasks required in order to implement a Cartoon Filter. From an implementation point of view, consuming code is only required to invoke this method, no other additional method calls are required. The definition as follows:

public static Bitmap CartoonEffectFilter( 
                                this Bitmap sourceBitmap, 
                                byte threshold = 0, 
                                SmoothingFilterType smoothFilter  
                                = SmoothingFilterType.None) 
{ 
    sourceBitmap = sourceBitmap.SmoothingFilter(smoothFilter); 

       
    BitmapData sourceData = 
               sourceBitmap.LockBits(new Rectangle (0, 0, 
               sourceBitmap.Width, sourceBitmap.Height), 
               ImageLockMode.ReadOnly, 
               PixelFormat.Format32bppArgb); 

   
    byte[] pixelBuffer = new byte[sourceData.Stride * 
                                  sourceData.Height]; 

   
    byte[] resultBuffer = new byte[sourceData.Stride * 
                                   sourceData.Height]; 

   
    Marshal.Copy(sourceData.Scan0, pixelBuffer, 0, 
                               pixelBuffer.Length); 

   
    sourceBitmap.UnlockBits(sourceData); 

   
    int byteOffset = 0; 
    int blueGradient, greenGradient, redGradient = 0; 
    double blue = 0, green = 0, red = 0; 

   
    bool exceedsThreshold = false; 

   
    for (int offsetY = 1; offsetY <  
         sourceBitmap.Height - 1; offsetY++) 
    {
        for (int offsetX = 1; offsetX < 
            sourceBitmap.Width - 1; offsetX++) 
        { 
            byteOffset = offsetY * sourceData.Stride + 
                         offsetX * 4; 

   
            blueGradient =  
            Math.Abs(pixelBuffer[byteOffset - 4] - 
            pixelBuffer[byteOffset + 4]); 

   
            blueGradient +=  
            Math.Abs(pixelBuffer[byteOffset - sourceData.Stride] -  
            pixelBuffer[byteOffset + sourceData.Stride]); 

               
            byteOffset++; 

   
            greenGradient =  
            Math.Abs(pixelBuffer[byteOffset - 4] -  
            pixelBuffer[byteOffset + 4]); 

   
            greenGradient +=  
            Math.Abs(pixelBuffer[byteOffset - sourceData.Stride] -  
            pixelBuffer[byteOffset + sourceData.Stride]); 

   
            byteOffset++; 

   
            redGradient =  
            Math.Abs(pixelBuffer[byteOffset - 4] -  
            pixelBuffer[byteOffset + 4]); 

   
            redGradient +=  
            Math.Abs(pixelBuffer[byteOffset - sourceData.Stride] -  
            pixelBuffer[byteOffset + sourceData.Stride]); 

               
            if (blueGradient + greenGradient + redGradient > threshold) 
            { exceedsThreshold = true ; } 
            else 
            { 
                byteOffset -= 2; 

   
                blueGradient = Math.Abs(pixelBuffer[byteOffset - 4] -  
                                        pixelBuffer[byteOffset + 4]); 
                byteOffset++; 

   
                greenGradient = Math.Abs(pixelBuffer[byteOffset - 4] -  
                                         pixelBuffer[byteOffset + 4]); 
                byteOffset++; 

   
                redGradient = Math.Abs(pixelBuffer[byteOffset - 4] -  
                                       pixelBuffer[byteOffset + 4]); 

   
                if (blueGradient + greenGradient + redGradient > threshold) 
                { exceedsThreshold = true ; } 
                else 
                { 
                    byteOffset -= 2; 

   
                    blueGradient =  
                    Math.Abs(pixelBuffer[byteOffset - sourceData.Stride] -  
                    pixelBuffer[byteOffset + sourceData.Stride]); 

   
                    byteOffset++; 

   
                    greenGradient =  
                    Math.Abs(pixelBuffer[byteOffset - sourceData.Stride] -  
                    pixelBuffer[byteOffset + sourceData.Stride]); 

   
                    byteOffset++; 

   
                    redGradient =  
                    Math.Abs(pixelBuffer[byteOffset - sourceData.Stride] -  
                    pixelBuffer[byteOffset + sourceData.Stride]); 

   
                    if (blueGradient + greenGradient +  
                              redGradient > threshold) 
                    { exceedsThreshold = true ; } 
                    else 
                    { 
                        byteOffset -= 2; 

   
                        blueGradient =  
                        Math.Abs(pixelBuffer[byteOffset - 4 - sourceData.Stride] -  
                        pixelBuffer[byteOffset + 4 + sourceData.Stride]); 

   
                        blueGradient +=  
                        Math.Abs(pixelBuffer[byteOffset - sourceData.Stride + 4] -  
                        pixelBuffer[byteOffset + sourceData.Stride - 4]); 

   
                        byteOffset++; 

   
                        greenGradient =  
                        Math.Abs(pixelBuffer[byteOffset - 4 - sourceData.Stride] -  
                        pixelBuffer[byteOffset + 4 + sourceData.Stride]); 

   
                        greenGradient +=  
                        Math.Abs(pixelBuffer[byteOffset - sourceData.Stride + 4] -  
                        pixelBuffer[byteOffset + sourceData.Stride - 4]); 

   
                        byteOffset++; 

   
                        redGradient =  
                        Math.Abs(pixelBuffer[byteOffset - 4 - sourceData.Stride] -  
                        pixelBuffer[byteOffset + 4 + sourceData.Stride]); 

   
                        redGradient += 
                        Math.Abs(pixelBuffer[byteOffset - sourceData.Stride + 4] -  
                        pixelBuffer[byteOffset + sourceData.Stride - 4]); 

   
                        if (blueGradient + greenGradient + redGradient > threshold) 
                        { exceedsThreshold = true ; } 
                        else 
                        { exceedsThreshold = false ; } 
                    } 
                } 
            }

   
            byteOffset -= 2; 

   
            if (exceedsThreshold) 
            { 
                blue = 0; 
                green = 0; 
                red = 0; 
            } 
            else 
            { 
                blue = pixelBuffer[byteOffset]; 
                green = pixelBuffer[byteOffset + 1]; 
                red = pixelBuffer[byteOffset + 2]; 
            } 

   
            blue = (blue > 255 ? 255 : 
                   (blue < 0 ? 0 : 
                    blue)); 

   
            green = (green > 255 ? 255 : 
                    (green < 0 ? 0 : 
                     green)); 

   
            red = (red > 255 ? 255 : 
                  (red < 0 ? 0 : 
                   red));  

   
            resultBuffer[byteOffset] = (byte)blue; 
            resultBuffer[byteOffset + 1] = (byte)green; 
            resultBuffer[byteOffset + 2] = (byte)red; 
            resultBuffer[byteOffset + 3] = 255;   
        } 
    } 

   
    Bitmap resultBitmap = new Bitmap(sourceBitmap.Width, 
                                     sourceBitmap.Height); 

   
    BitmapData resultData = 
               resultBitmap.LockBits(new Rectangle(0, 0, 
               resultBitmap.Width, resultBitmap.Height), 
               ImageLockMode.WriteOnly, 
               PixelFormat.Format32bppArgb); 

   
    Marshal.Copy(resultBuffer, 0, resultData.Scan0, 
                               resultBuffer.Length); 

   
    resultBitmap.UnlockBits(resultData); 

   
    return resultBitmap; 
}

Sample Images

The sample image used in this article which features Bill Gates has been licensed under the Creative Commons Attribution 2.0 Generic license and can be downloaded from Wikipedia.

The sample image featuring Steve Ballmer has been licensed under the Creative Commons Attribution 2.0 Generic license and can be downloaded from Wikipedia.

The sample image featuring an Amber flush Rose has been licensed under the Creative Commons Attribution-Share Alike 3.0 Unported, 2.5 Generic, 2.0 Generic and 1.0 Generic license and can be downloaded from Wikipedia.

The sample image featuring a Computer Processor has been licensed under the Creative Commons Attribution-Share Alike 2.0 Generic license and can be downloaded from Wikipedia.l The original author is attributed as Andrew Dunn – http://www.andrewdunnphoto.com/

The Original Image

No Smoothing, Threshold 100

Gaussian 3×3, Threshold 73

Gaussian 5×5, Threshold 78

Gaussian 7×7, Threshold 84

Low Pass 3×3, Threshold 72

Low Pass 5×5, Threshold 81

Mean 3×3, Threshold 79

Mean 5×5, Threshold 80

Median 3×3, Threshold 85

Median 5×5, Threshold 105

Median 7×7, Threshold 127

Median 9×9, Threshold 154

Sharpen 3×3, Threshold 114

C# How to: Gradient Based Edge Detection

Published June 1, 2013 Augmented Reality , C# , Code Samples , Edge Detection , Extension Methods , Graphic Filters , Graphics , How to , Image Convolution , Image Filters , Learn Everyday , Microsoft , Morphology Filters , New Version , Opensource , Update 31 Comments
Tags: Augmented Reality, Binary Image, Bitmap ARGB, Bitmap Filters, Bitmap.LockBits, BitmapData, Boolean Edge Detection, C#, Change detection, Code Sample, Computer vision, Converting Images, Convolution filters, Convolution Kernel, Convolution matrix, Edge Masks, Feature extraction, Gradient Edge Detection, Graphic Filters, Image, Image Edge Detection, Image Gradient, Image Morphology, Image processing, Image Sharpen, Image Sharpness, Image Transform, Line Detection, Machine vision, Matrix, Pixel Filters

Article purpose

This article provides a technical discussion exploring the topic of Gradient Based Edge Detection and related aspects. Several filtering options are illustrated and explained ranging from pure black and white edge detection to image sharpening.

Sample Source Code

This article is accompanied by a sample source code Visual Studio project which is available for download here.

Using the Sample Application

All of the concepts implemented in this article can be replicated and tested by making use of the sample application included in the associated sample source code. The sample application user interface provides several configurable options to be implemented when performing Gradient Based Edge Detection. The available configuration categories are: Filter Type, Derivative Level, Threshold and Colour Factor Filters.

Configurable Filter Types exposed to the end user consist of:

None – When selecting this option no filtering will be applied. Source/input images are displayed reflecting no change.
Edge Detect Mono – This option represents basic Gradient Based Edge Detection. Resulting images are only expressed in terms of black and white pixels.
Edge Detect Gradient – Gradient Based Edge Detection revolves around calculating pixel colour gradients. This option signifies a scenario where the pixels forming resulting images express the relevant pixel’s colour gradient, when a pixel has been determined to reflect part of an edge. If a pixel is not considered to be part of an edge, the relevant pixel’s colour value will be set to black.
Sharpen – In terms of edge detection, image sharpening can be achieved by emphasising detected edges in source/input images. Emphasising edges involves combining a source/input image and an image which express only detected edges.
Sharpen Gradient – This option combines calculated colour gradients and the original colour value of a pixel on a per pixel basis when a pixel has been determined to be part of an edge. If a pixel does not form part of an edge, the pixel’s colour value is set to that of the original pixel colour.

The user interface defines two RadioButtons: First Derivative and Second Derivative. These user interface options relate to the edge detection method being implemented, either First Order or Second Order derivative operators.

Comparing a global threshold and colour gradients on a per pixel scenario forms the basis of Gradient Based Edge Detection. The TrackBar labelled Threshold enables the user to adjust the global threshold value implemented in pixel colour gradient comparisons.

The Colour Factor Filters impact on the level or extent to which colours are expressed in resulting images. The three colour factors, Red, Green and Blue are intended to be used in combination with the filtering options: Filter Type and Threshold. Colour Factor Filter affects when implemented in combination:

Filter Type – Edge Detect Mono: Not applicable. Edge Detect Mono filtering discards all pixel colour data.
Filter Type – Edge Detect Gradient: If a pixel is detected as part of an edge, the pixel’s colour values will be set to the gradient calculated when evaluating edge criteria. Gradient values are multiplied by Colour Factor values before being assigned to a resulting image pixel.
Filter Type – Sharpen: The pixels which form part of an edge, in terms of the resulting image the corresponding pixels will be set to the same colour values. In addition pixel colour values in the resulting image are multiplied by with the relevant Colour Factor. The image pixels not detected as part of an edge will not be multiplied with any Colour Factor values.
Filter Type – Sharpen Gradient: Edge detected pixels in a source/input image will have the effect of corresponding pixels in the resulting image being assigned to the calculated colour gradient value, multiplied with the relevant Colour Factor value. In the scenario of a pixel not being detected as forming part of an edge, Colour Factors will not be implemented.
Threshold: The global threshold value specified by the user determines the level of sensitivity to which edges will be detected. The degree to which edges are detected through the threshold value impacts upon whether a pixel will be multiplied with the relevant Colour Factor value.

The user has the option of saving filtered image to the local file system by clicking the Save Image button. The image below is a screenshot of the Gradient Based Edge Detection sample application in action:

Gradient Based Edge Detection Theory

Gradient Based Edge Detection qualifies to be classified as a neighbouring pixel algorithm. When calculating a pixel’s value in order to determine if a pixel should be expressed as part of an edge or not, the result will be determined by:

The values expressed by neighbouring pixels. The more intense or sudden differences that occur between neighbouring pixels will result in higher accuracy edge detection.
A user specified global threshold value used in comparison operations acts as a cut-off value, ultimately being the final factor to determine if a pixel should be expressed as part of an edge.

In the sample source code we implement the following steps when calculating whether a pixel should be considered as part of an edge:

Iterate each pixel that forms part of the source/input image.
Calculate and combine horizontal and vertical gradients for each of the colour components Red, Green and Blue. If the sum total of each colour component’s calculated gradient exceeds the global threshold value consider the pixel being iterated as part of an edge. If the sum total of colour gradients equate to less than the global threshold implement step 3.
Calculate a pixel’s horizontal gradient per colour component. When comparing the gradient sum total against the global threshold consider the pixel being iterated part of an edge if the sum total of gradient values exceed that of the threshold. If the total of colour gradient value do not exceed the threshold value continue to step 4.
Calculate a pixel’s vertical gradient per colour component. When comparing the gradient sum total against the global threshold consider the pixel being iterated part of an edge if the sum total of gradient values exceed that of the threshold. If the sum total of colour gradients do not exceed the threshold value continue to step 5.
Calculate and combine diagonal gradients for each of the colour components Red, Green and Blue. If the sum total of each colour component’s calculated gradient exceeds the global threshold value consider the pixel being iterated as being part of an edge. If the sum total of colour gradients equate to less than the global threshold the pixel being iterated should not be considered as part of an edge.

If we determined that a pixel forms part of an edge, the value expressed by the corresponding pixel in the resulting image will be determined by the Image Filter configuration value:

Edge Detect Mono – All pixels will be set to white.
Edge Detect Gradient – Each colour component will be assigned to the related colour gradient calculated when performing edge detection. Each Colour gradient will be multiplied with the related colour factor.
Sharpen – The value of a resulting pixel will be calculated as the product of the corresponding source pixel and the related colour factor value.
Sharpen Gradient – Results are calculated in terms of the sum total of the corresponding input pixel and the product of the related colour gradient and colour factor value.

Implementing Gradient Based Edge Detection

The sample source code associated with this article provides the defines of the GradientBasedEdgeDetectionFilter extension method targeting the Bitmap class. This method iterates every pixel contained in the source/input image. Whilst iterating pixels the method creates a 3×3 window/kernel covering the neighbouring pixels surrounding the pixel currently being iterated. Colour gradients are calculated from every pixel’s neighbouring pixels.

The following Code snippet provides the definition of the GradientBasedEdgeDetectionFilter extension method:

public static Bitmap GradientBasedEdgeDetectionFilter( 
                                this Bitmap sourceBitmap, 
                                EdgeFilterType filterType, 
                                DerivativeLevel derivativeLevel,  
                                float redFactor = 1.0f, 
                                float greenFactor = 1.0f, 
                                float blueFactor = 1.0f, 
                                byte threshold = 0) 
{ 
    BitmapData sourceData = 
               sourceBitmap.LockBits(new Rectangle (0, 0, 
               sourceBitmap.Width, sourceBitmap.Height), 
               ImageLockMode.ReadOnly, 
               PixelFormat.Format32bppArgb); 

   
    byte[] pixelBuffer = new byte[sourceData.Stride * 
                                  sourceData.Height]; 

   
    byte[] resultBuffer = new byte[sourceData.Stride * 
                                   sourceData.Height]; 

   
    Marshal.Copy(sourceData.Scan0, pixelBuffer, 0, 
                               pixelBuffer.Length); 

   
    sourceBitmap.UnlockBits(sourceData); 

   
    int derivative = (int)derivativeLevel; 
    int byteOffset = 0; 
    int blueGradient, greenGradient, redGradient = 0; 
    double blue = 0, green = 0, red = 0; 

   
    bool exceedsThreshold = false; 

   
    for(int offsetY = 1; offsetY < sourceBitmap.Height - 1; offsetY++) 
    {
        for (int offsetX = 1; offsetX < 
            sourceBitmap.Width - 1; offsetX++) 
        { 
            byteOffset = offsetY * sourceData.Stride + 
                         offsetX * 4; 

   
            blueGradient =  
            Math.Abs(pixelBuffer[byteOffset - 4] - 
            pixelBuffer[byteOffset + 4]) / derivative; 

   
            blueGradient +=  
            Math.Abs(pixelBuffer[byteOffset - sourceData.Stride] -  
            pixelBuffer[byteOffset + sourceData.Stride]) / derivative; 

               
            byteOffset++; 

   
            greenGradient =  
            Math.Abs(pixelBuffer[byteOffset - 4] -  
            pixelBuffer[byteOffset + 4]) / derivative; 

   
            greenGradient +=  
            Math.Abs(pixelBuffer[byteOffset - sourceData.Stride] -  
            pixelBuffer[byteOffset + sourceData.Stride]) / derivative; 

   
            byteOffset++; 

   
            redGradient =  
            Math.Abs(pixelBuffer[byteOffset - 4] -  
            pixelBuffer[byteOffset + 4]) / derivative; 

   
            redGradient +=  
            Math.Abs(pixelBuffer[byteOffset - sourceData.Stride] -  
            pixelBuffer[byteOffset + sourceData.Stride]) / derivative; 

               
            if (blueGradient + greenGradient + redGradient > threshold) 
            { exceedsThreshold = true ; } 
            else 
            { 
                byteOffset -= 2; 

   
                blueGradient = Math.Abs(pixelBuffer[byteOffset - 4] -  
                                        pixelBuffer[byteOffset + 4]); 
                byteOffset++; 

   
                greenGradient = Math.Abs(pixelBuffer[byteOffset - 4] -  
                                         pixelBuffer[byteOffset + 4]); 
                byteOffset++; 

   
                redGradient = Math.Abs(pixelBuffer[byteOffset - 4] -  
                                       pixelBuffer[byteOffset + 4]); 

   
                if (blueGradient + greenGradient + redGradient > threshold) 
                { exceedsThreshold = true ; } 
                else 
                { 
                    byteOffset -= 2; 

   
                    blueGradient =  
                    Math.Abs(pixelBuffer[byteOffset - sourceData.Stride] -  
                    pixelBuffer[byteOffset + sourceData.Stride]); 

   
                    byteOffset++; 

   
                    greenGradient =  
                    Math.Abs(pixelBuffer[byteOffset - sourceData.Stride] -  
                    pixelBuffer[byteOffset + sourceData.Stride]); 

   
                    byteOffset++; 

   
                    redGradient =  
                    Math.Abs(pixelBuffer[byteOffset - sourceData.Stride] -  
                    pixelBuffer[byteOffset + sourceData.Stride]); 

   
                    if (blueGradient + greenGradient +  
                              redGradient > threshold) 
                    { exceedsThreshold = true ; } 
                    else 
                    { 
                        byteOffset -= 2; 

   
                        blueGradient =  
                        Math.Abs(pixelBuffer[byteOffset - 4 - sourceData.Stride] -  
                        pixelBuffer[byteOffset + 4 + sourceData.Stride]) / derivative; 

   
                        blueGradient +=  
                        Math.Abs(pixelBuffer[byteOffset - sourceData.Stride + 4] -  
                        pixelBuffer[byteOffset + sourceData.Stride - 4]) / derivative; 

   
                        byteOffset++; 

   
                        greenGradient =  
                        Math.Abs(pixelBuffer[byteOffset - 4 - sourceData.Stride] -  
                        pixelBuffer[byteOffset + 4 + sourceData.Stride]) / derivative; 

   
                        greenGradient +=  
                        Math.Abs(pixelBuffer[byteOffset - sourceData.Stride + 4] -  
                        pixelBuffer[byteOffset + sourceData.Stride - 4]) / derivative; 

   
                        byteOffset++; 

   
                        redGradient =  
                        Math.Abs(pixelBuffer[byteOffset - 4 - sourceData.Stride] -  
                        pixelBuffer[byteOffset + 4 + sourceData.Stride]) / derivative; 

   
                        redGradient += 
                        Math.Abs(pixelBuffer[byteOffset - sourceData.Stride + 4] -  
                        pixelBuffer[byteOffset + sourceData.Stride - 4]) / derivative; 

   
                        if (blueGradient + greenGradient + redGradient > threshold) 
                        { exceedsThreshold = true ; } 
                        else 
                        { exceedsThreshold = false ; } 
                    } 
                } 
            } 

   
            byteOffset -= 2; 

   
            if (exceedsThreshold) 
            { 
                if (filterType == EdgeFilterType.EdgeDetectMono) 
                { 
                    blue = green = red = 255; 
                } 
                else if (filterType == EdgeFilterType.EdgeDetectGradient) 
                { 
                    blue = blueGradient * blueFactor; 
                    green = greenGradient * greenFactor; 
                    red = redGradient * redFactor; 
                } 
                else if (filterType == EdgeFilterType.Sharpen) 
                {
                    blue = pixelBuffer[byteOffset] * blueFactor; 
                    green = pixelBuffer[byteOffset + 1] * greenFactor; 
                    red = pixelBuffer[byteOffset + 2] * redFactor; 
                } 
                else if (filterType == EdgeFilterType.SharpenGradient) 
                { 
                    blue = pixelBuffer[byteOffset] + blueGradient * blueFactor; 
                    green = pixelBuffer[byteOffset + 1] + greenGradient * greenFactor; 
                    red = pixelBuffer[byteOffset + 2] + redGradient * redFactor; 
                } 
            } 
            else 
            { 
                if (filterType == EdgeFilterType.EdgeDetectMono ||  
                    filterType == EdgeFilterType.EdgeDetectGradient) 
                { 
                    blue = green = red = 0; 
                } 
                else if (filterType == EdgeFilterType.Sharpen ||  
                         filterType == EdgeFilterType.SharpenGradient) 
                { 
                    blue = pixelBuffer[byteOffset]; 
                    green = pixelBuffer[byteOffset + 1]; 
                    red = pixelBuffer[byteOffset + 2]; 
                } 
            } 

   
            blue = (blue > 255 ? 255 : 
                   (blue < 0 ? 0 : 
                    blue)); 

   
            green = (green > 255 ? 255 : 
                    (green < 0 ? 0 : 
                     green)); 

   
            red = (red > 255 ? 255 : 
                  (red < 0 ? 0 : 
                   red));  

   
            resultBuffer[byteOffset] = (byte)blue; 
            resultBuffer[byteOffset + 1] = (byte)green; 
            resultBuffer[byteOffset + 2] = (byte)red; 
            resultBuffer[byteOffset + 3] = 255;   
        } 
    } 
 
    Bitmap resultBitmap = new Bitmap(sourceBitmap.Width, 
                                     sourceBitmap.Height); 

   
    BitmapData resultData = 
               resultBitmap.LockBits(new Rectangle (0, 0, 
               resultBitmap.Width, resultBitmap.Height), 
               ImageLockMode.WriteOnly, 
               PixelFormat.Format32bppArgb); 

   
    Marshal.Copy(resultBuffer, 0, resultData.Scan0, 
                               resultBuffer.Length); 

   
    resultBitmap.UnlockBits(resultData); 

   
    return resultBitmap; 
}

Sample Images

The banner images depicting a butterfly featured throughout this article were generated using the sample application. The original image has been licenced under the Creative Commons Attribution-Share Alike 3.0 Unported, 2.5 Generic, 2.0 Generic and 1.0 Generic license. The original image is attributed to Kenneth Dwain Harrelson and can be downloaded from Wikipedia.

The sample image featuring a Scarlet Macaw has been licensed under the Creative Commons Attribution-Share Alike 3.0 Germany license. The original image can be downloaded from Wikipedia.

The Original Image

Edge Detect, Second Derivative, Threshold 50

Edge Detect Gradient, First Derivative, Blue

Edge Detect Gradient, First Derivative, Green

Edge Detect Gradient, First Derivative, Green and Blue

Edge Detect Gradient, First Derivative, Red

Edge Detect Gradient, First Derivative, Red and Blue

Edge Detect Gradient, First Derivative, Red and Green

Edge Detect Gradient, First Derivative, Red, Green and Blue

Edge Detect Sharpen, Second Derivative, Threshold 40, Black

Edge Detect Sharpen, Second Derivative, Threshold 40, Blue

Edge Detect Sharpen, Second Derivative, Threshold 40, Green

Edge Detect Sharpen, Second Derivative, Threshold 40, Green and Blue

Edge Detect Sharpen, Second Derivative, Threshold 40, Red

Edge Detect Sharpen, Second Derivative, Threshold 40, Red and Blue

Edge Detect Sharpen, Second Derivative, Threshold 40, Red and Green

Edge Detect Sharpen, Second Derivative, Threshold 40, White

Edge Detect Sharpen Gradient, First Derivative, Threshold 0, Blue

Edge Detect Sharpen Gradient, First Derivative, Threshold 0, Green

Edge Detect Sharpen Gradient, First Derivative, Threshold 0, Green and Blue

Edge Detect Sharpen Gradient, First Derivative, Threshold 0, Red

Edge Detect Sharpen Gradient, First Derivative, Threshold 0, Red and Blue

Edge Detect Sharpen Gradient, First Derivative, Threshold 0, Red and Green

Edge Detect Sharpen Gradient, First Derivative, Threshold 0, White

C# How to: Boolean Edge Detection

Published June 1, 2013 Augmented Reality , C# , Code Samples , Edge Detection , Extension Methods , Graphic Filters , Graphics , How to , Image Filters , Learn Everyday , Microsoft , Morphology Filters , New Version , Opensource , Update 31 Comments
Tags: Augmented Reality, Binary Image, Bitmap ARGB, Bitmap Filters, Bitmap.LockBits, BitmapData, Boolean Edge Detection, C#, Change detection, Code Sample, Computer vision, Converting Images, Convolution filters, Convolution Kernel, Convolution matrix, Edge Masks, Feature extraction, Graphic Filters, Image, Image Edge Detection, Image Morphology, Image processing, Image Transform, Line Detection, Machine vision, Matrix, Pixel Filters

Article purpose

The purpose of this article is to detail Boolean Function Based Edge Detection. The Image filtering implemented in this article occurs on a per pixel basis. The implementation relies on linear algebra. No GDI+ or traditional drawing methods are required.

Sample Source Code

This article is accompanied by a sample source code Visual Studio project which is available for download here.

Using the Sample Application

Implemented as part of this article’s sample source code is a Sample Application. The concepts detailed in this article have all been implemented and tested using the associated Sample Application.

The first task required in using the Sample Application comes in the form of having to specify a source/input image. Select image files from the local file system by clicking the Load Image button.

On the right-hand side of the Sample Application’s user interface the user will be presented with a set of controls which relate to various filter options. Users are able to specify Edge Detection implementation methods when adjusting filter options.

Filter type values are: None, Edge Detect and Sharpen. Selecting a filter type of None results in no filtering being implemented, the original input/source image will be displayed reflecting no change. When users select the Edge Detect filter type the resulting output image reflects a black and white image of which only the detected edges are visible. The Sharpen filter type implements Boolean Edge Detection producing a sharpened image by means of highlighting detected edges within the original input/source image.

The Trackbar labelled Threshold is intended to allow users the option of reducing image noise expressed as detected edges in the resulting image. The level of image noise present differs depending on the input/source image specified, hence the option of implementing a threshold.

The three remaining TrackBar controls are labelled Red, Green and Blue. The Colour Factor filter options allows the user to specify the extend to which detected edges are expressed when sharpening an image. When all three factor values are set to the same value edges appear white if the factor value exceeds zero. Factor values set to zero results in detected edges appearing as darker/black edge lines. Factor values can be set per colour value, which will have the effect of creating a coloured outline being visible in the result image. The colour of the outlining effect can be controlled by adjusting individual colour factor values.

After having implemented image filtering the user has the option of saving the result image to the local file system by clicking the Save Image button. The image shown below is screenshot of the Boolean Edge Detection sample application in action:

The Local Threshold and Boolean Function Based Edge Detection

Boolean Edge Detection is considered a a subset of Image Morphological Filtering. This method of edge detection employs both a local and global threshold. Implementation of the Boolean Edge Detection algorithm can be achieved by completing the following steps:

Initiate a process of iterating each pixel that forms part of the source/input image. Calculate a local threshold value based on a 3×3 matrix/window. The matrix should be positioned in a fashion where the pixel currently being iterated is located in the middle of the matrix. Calculate a mean value using as input the 9 values covered by the matrix. Create a new blank matrix set to 3×3 dimensions. Compare each pixel in the source image to the calculated mean value. If a pixel’s value exceeds that of the mean value set the corresponding location on the blank matrix to one. If a pixel’s value does not exceed that of the mean value set the corresponding location on the blank matrix to zero.
In the next step compare the newly created matrix to the set of 16 edge masks. If the new matrix is represented in the edge masks the middle pixel being iterated should be set to indicate an edge.
The first two steps have to be repeated for each pixel contained in the source image, in other words each pixel should be iterated. Edges should now be detected as you progress through image pixels, although false edges will also be present as a result of image noise.
False edges that were detected can be removed when implementing a global threshold. Firstly calculate the variance of each 3×3 matrix. If the pixel currently being iterated was detected as part of an edge in step 2 and the variance calculated exceeds the global threshold the pixel can be considered as part of an edge. If the variance calculated equates to less than the global threshold a pixel does not form part of an edge even if the calculated matrix matches one of the 16 edge masks.

The following image illustrates the 16 edge masks:

Implementing Boolean Edge Detection

The sample source code implements the BooleanEdgeDetectionFilter extension method targeting the Bitmap class. This method implements the Boolean Edge Detection theoretical steps discussed in the previous section.

In order to determine if a newly calculated matrix, as described in step 1, matches any of the 16 pre-defined edge masks the BooleanEdgeDetectionFilter extension method implements string comparison. The reasoning behind string based edge mask comparison boils down to efficiency, both in terms of reducing code complexity and improving performance. The method defines a generic List of type string and then proceeds to add 16 strings, each representing an edge mask. Edge mask strings express an edge mask matrix in terms of a row and column format. The following code snippet lists the 16 edge masks strings being defined:

List<string> edgeMasks = new List<string>();

 
edgeMasks.Add("011011011");
edgeMasks.Add("000111111");
edgeMasks.Add("110110110");
edgeMasks.Add("111111000");
edgeMasks.Add("011011001");
edgeMasks.Add("100110110");
edgeMasks.Add("111011000");
edgeMasks.Add("111110000");
edgeMasks.Add("111011001");
edgeMasks.Add("100110111");
edgeMasks.Add("001011111");
edgeMasks.Add("111110100");
edgeMasks.Add("000011111");
edgeMasks.Add("000110111");
edgeMasks.Add("001011011");
edgeMasks.Add("001011011");
edgeMasks.Add("110110100");

The following code snippet list the complete implementation of the BooleanEdgeDetectionFilter extension method:

public static Bitmap BooleanEdgeDetectionFilter( 
                                this Bitmap sourceBitmap, 
                                BooleanFilterType filterType, 
                                float redFactor = 1.0f, 
                                float greenFactor = 1.0f, 
                                float blueFactor = 1.0f, 
                                byte threshold = 0) 
{ 
    BitmapData sourceData = 
               sourceBitmap.LockBits(new Rectangle(0, 0, 
               sourceBitmap.Width, sourceBitmap.Height), 
               ImageLockMode.ReadOnly, 
               PixelFormat.Format32bppArgb); 

 
    byte[] pixelBuffer = new byte[sourceData.Stride * 
                                  sourceData.Height]; 
 
    byte[] resultBuffer = new byte[sourceData.Stride * 
                                   sourceData.Height]; 

 
    Marshal.Copy(sourceData.Scan0, pixelBuffer, 0, 
                               pixelBuffer.Length); 

 
    sourceBitmap.UnlockBits(sourceData); 

 
    List<string> edgeMasks = new List<string>(); 

 
    edgeMasks.Add("011011011"); 
    edgeMasks.Add("000111111"); 
    edgeMasks.Add("110110110"); 
    edgeMasks.Add("111111000"); 
    edgeMasks.Add("011011001"); 
    edgeMasks.Add("100110110"); 
    edgeMasks.Add("111011000"); 
    edgeMasks.Add("111110000"); 
    edgeMasks.Add("111011001"); 
    edgeMasks.Add("100110111"); 
    edgeMasks.Add("001011111"); 
    edgeMasks.Add("111110100"); 
    edgeMasks.Add("000011111"); 
    edgeMasks.Add("000110111"); 
    edgeMasks.Add("001011011"); 
    edgeMasks.Add("001011011"); 
    edgeMasks.Add("110110100"); 

 
    int filterOffset = 1; 
    int calcOffset = 0; 

 
    int byteOffset = 0; 
    int matrixMean = 0; 
    int matrixTotal = 0; 
    double matrixVariance = 0; 

 
    double blueValue = 0; 
    double greenValue = 0; 
    double redValue = 0; 

 
    string matrixPatern = String.Empty; 

 
    for (int offsetY = filterOffset; offsetY < 
        sourceBitmap.Height - filterOffset; offsetY++) 
    { 
        for (int offsetX = filterOffset; offsetX < 
            sourceBitmap.Width - filterOffset; offsetX++) 
        {
            byteOffset = offsetY * 
                         sourceData.Stride + 
                         offsetX * 4; 

 
            matrixMean = 0; 
            matrixTotal = 0; 
            matrixVariance = 0; 

 
            matrixPatern = String.Empty; 

 
            //Step 1: Calculate local matrix  
            for (int filterY = -filterOffset; 
                filterY <= filterOffset; filterY++) 
            { 
                for (int filterX = -filterOffset; 
                    filterX <= filterOffset; filterX++) 
                { 
                    calcOffset = byteOffset + 
                                 (filterX * 4) + 
                    (filterY * sourceData.Stride); 

 
                    matrixMean += pixelBuffer[calcOffset]; 
                    matrixMean += pixelBuffer[calcOffset + 1]; 
                    matrixMean += pixelBuffer[calcOffset + 2]; 
                }
            }

 
            matrixMean = matrixMean / 9; 

 
            //Step 4: Calculate Variance  
            for (int filterY = -filterOffset; 
                filterY <= filterOffset; filterY++) 
            { 
                for (int filterX = -filterOffset; 
                    filterX <= filterOffset; filterX++) 
                { 
                    calcOffset = byteOffset + 
                                 (filterX * 4) + 
                    (filterY * sourceData.Stride); 

 
                    matrixTotal = pixelBuffer[calcOffset]; 
                    matrixTotal += pixelBuffer[calcOffset+1]; 
                    matrixTotal += pixelBuffer[calcOffset+2]; 

 
                    matrixPatern += (matrixTotal > matrixMean  
                                                 ? "1"  : "0" ); 

 
                    matrixVariance +=  
                        Math.Pow(matrixMean -  
                        (pixelBuffer[calcOffset] +  
                        pixelBuffer[calcOffset + 1] +  
                        pixelBuffer[calcOffset + 2]), 2); 
                } 
            } 

 
            matrixVariance = matrixVariance / 9; 

 
            if (filterType == BooleanFilterType.Sharpen) 
            { 
                blueValue = pixelBuffer[byteOffset]; 
                greenValue = pixelBuffer[byteOffset + 1]; 
                redValue = pixelBuffer[byteOffset + 2]; 

 
                //Step 4: Exlclude noise using global  
                // threshold  
                if (matrixVariance > threshold) 
                {   //Step 2: Compare newly calculated  
                    // matrix and image masks  
                    if (edgeMasks.Contains(matrixPatern)) 
                    {
                        blueValue = (blueValue * blueFactor); 
                        greenValue = (greenValue * greenFactor); 
                        redValue = (redValue * redFactor); 

 
                        blueValue = (blueValue > 255 ? 255 : 
                                    (blueValue < 0 ? 0 : 
                                     blueValue)); 

 
                        greenValue = (greenValue > 255 ? 255 : 
                                     (greenValue < 0 ? 0 : 
                                      greenValue)); 

 
                        redValue = (redValue > 255 ? 255 : 
                                   (redValue < 0 ? 0 : 
                                    redValue)); 
                    } 
                }
            }    //Step 4: Exlclude noise using global  
                 // threshold  
                 //Step 2: Compare newly calculated  
                 // matrix and image masks  
            else  if (matrixVariance > threshold &&  
                     edgeMasks.Contains(matrixPatern)) 
            { 
                blueValue = 255; 
                greenValue = 255; 
                redValue = 255; 
            }
            else  
            { 
                blueValue = 0; 
                greenValue = 0; 
                redValue = 0; 
            } 

 
            resultBuffer[byteOffset] = (byte)blueValue; 
            resultBuffer[byteOffset + 1] = (byte)greenValue; 
            resultBuffer[byteOffset + 2] = (byte)redValue; 
            resultBuffer[byteOffset + 3] = 255;   
        }
    } 

 
    Bitmap resultBitmap = new Bitmap(sourceBitmap.Width, 
                                     sourceBitmap.Height); 

 
    BitmapData resultData = 
               resultBitmap.LockBits(new Rectangle(0, 0, 
               resultBitmap.Width, resultBitmap.Height), 
               ImageLockMode.WriteOnly, 
               PixelFormat.Format32bppArgb); 

 
    Marshal.Copy(resultBuffer, 0, resultData.Scan0, 
                               resultBuffer.Length); 

 
    resultBitmap.UnlockBits(resultData); 

 
    return resultBitmap; 
}

Sample Images

This article features a photograph of The Eiffel Tower used in generating sample images. The original image has been licensed under the Creative Commons Attribution-Share Alike 3.0 Unported license and can be downloaded from Wikipedia: Original Image.

The Original Image

Edge Detection, Threshold 50

Sharpen, Threshold 50, Blue

Sharpen, Threshold 50, Green

Sharpen, Threshold 50, Green and Blue

Sharpen, Threshold 50, Red

Sharpen, Threshold 50, Red and Blue

Sharpen, Threshold 50, Red and Green

Sharpen, Threshold 50, White – Red, Green and Blue

Archive for the 'Morphology Filters' Category

Article Purpose

Sample Source Code

Using the Sample Application

Fuzzy Blur Overview

Mean Filter Blurring

Boolean Edge Detection without a local threshold

Implementing a Mean Filter

Implementing Boolean Edge Detection

Implementing a Fuzzy Blur Filter

Sample Images

Related Articles and Feedback

Rate this:

Share this:

Article Purpose

Sample Source Code

Using the Sample Application

Morphological Boundary Extraction

Boundary Sharpening

Boundary Tracing

Implementing Morphological Erosion and Dilation

Implementing Image Addition

Implementing Image Subtraction

Implementing Image Boundary Extraction

Implementing Image Boundary Sharpening

Implementing Image Boundary Tracing

Implementing a Wrapper Method

Sample Images

Related Articles and Feedback

Rate this:

Share this:

Article purpose

Sample source code

Using the Sample Application

Explanation of the Cartoon Effect

Implementing Cartoon Effects

Sample Images

Related Articles and Feedback

Rate this:

Share this:

Article purpose

Sample Source Code

Using the Sample Application

Gradient Based Edge Detection Theory

Implementing Gradient Based Edge Detection

Sample Images

Related Articles and Feedback

Rate this:

Share this:

Article purpose

Sample Source Code

Using the Sample Application

The Local Threshold and Boolean Function Based Edge Detection

Implementing Boolean Edge Detection

Sample Images

Related Articles and Feedback

Rate this:

Share this:

Dewald Esterhuizen

RSS Feed

Recent Posts

Blog Stats

Follow SoftwareByDefault via Email

Categories

Top Posts & Pages

Archives

C# How to Articles

SoftwareByDefault Online References

MSDN Articles

Categories

Tags