Call OpenCV functions from C#.net (Bitmap to Mat and Mat to Bitmap)


This is the second article of the article series which provide answers to following question! How to call OpenCV functions from C#.net or VB.net. Specially this article describes, how to pass System.Drawing.Bitmap to OpenCV and get a resultant image as System.Drawing.Bitmap from OpenCV.


Note that System.Drawing.Bitmap is the class type which allow you to manipulate images in C# while OpenCV treat images as cv::Mat (matrix). Therefore we need a way to convert from Bitmap to Mat vice versa in order to process and show processed images. This is the place where wrapper involved. For more details about wrappers, please refer previous article. 


From Previous Article...
So now we are going to create this wrapper for our application. Since we are dealing with .net framework, we can use CLR (Common Language Runtime) technique to create this wrapper. First you have to create a CLR project in Visual Studio. This post will describe, how to call Opencv functions from winfrom/C# and apply an Opencv filter to an image and show the Opencv window from winform.


Download complete Visual Studio project.

Step 1 - Create CLI Project

First of all we need to have a CLI project where we can call C++ functions from .net. You can follow the steps from previous article to create a CLI project.

Step 2 - Create converter function from Bitmap to Mat

Now we need to convert System.Drawing.Bitmap to cv::Mat. To do this conversion, we need to get to the primitive level of both data type. That's mean, we can simply think every image has created from a set of bytes. So bytes can live in both C++ and C#. Therefore the cv::Mat should be created from the set of bytes of Bitmap. Simply copy all bytes from Bitmap to Mat, finally it will create Mat from Bitmap. Use following function to do this conversion.

Mat BitmapToMat(System::Drawing::Bitmap^ bitmap)
{
    IplImage* tmp;

    System::Drawing::Imaging::BitmapData^ bmData = bitmap->LockBits(System::Drawing::Rectangle(0, 0, bitmap->Width, bitmap->Height), System::Drawing::Imaging::ImageLockMode::ReadWrite, bitmap->PixelFormat);
    if (bitmap->PixelFormat == System::Drawing::Imaging::PixelFormat::Format8bppIndexed)
    {
        tmp = cvCreateImage(cvSize(bitmap->Width, bitmap->Height), IPL_DEPTH_8U, 1);
        tmp->imageData = (char*)bmData->Scan0.ToPointer();
    }

    else if (bitmap->PixelFormat == System::Drawing::Imaging::PixelFormat::Format24bppRgb)
    {
        tmp = cvCreateImage(cvSize(bitmap->Width, bitmap->Height), IPL_DEPTH_8U, 3);
        tmp->imageData = (char*)bmData->Scan0.ToPointer();
    }

    bitmap->UnlockBits(bmData);

    return Mat(tmp);
}

Step 3 - Add System.Drawing namespace reference.

Once you add above function to your CLI project, you will get an error on System::Drawing::Bitmap. The reason for this error is that the project has no reference to System::Drawing::Bitmap. Follow below steps to add the reference.

Open project properties of CLI project.
















Select Common Properties then References , Click on Add New References... It will open a window that can select dll files where you can add as references.















Select Framework under Assembly category. Then mark System.Drawing.









Now the project has referenced System.Drawing, and you should not see any compile errors in the function.

Step 4 - Create converter function from Mat to Bitmap

Same as previous conversion, we need to perform this conversion from the primitive level. That's mean we need to reconstruct the Bitmap from Mat's image data bytes. Use following function to convert from cv::Mat to System.Drawing.Bitmap.

System::Drawing::Bitmap^ MatToBitmap(Mat srcImg){
    int stride = srcImg.size().width * srcImg.channels();//calc the srtide
    int hDataCount = srcImg.size().height;
   
    System::Drawing::Bitmap^ retImg;
       
    System::IntPtr ptr(srcImg.data);
   
    //create a pointer with Stride
    if (stride % 4 != 0){//is not stride a multiple of 4?
        //make it a multiple of 4 by fiiling an offset to the end of each row


       
//to hold processed data
        uchar *dataPro = new uchar[((srcImg.size().width * srcImg.channels() + 3) & -4) * hDataCount];

        uchar *data = srcImg.ptr();

        //current position on the data array
        int curPosition = 0;
        //current offset
        int curOffset = 0;

        int offsetCounter = 0;

        //itterate through all the bytes on the structure
        for (int r = 0; r < hDataCount; r++){
            //fill the data
            for (int c = 0; c < stride; c++){
                curPosition = (r * stride) + c;

                dataPro[curPosition + curOffset] = data[curPosition];
            }

            //reset offset counter
            offsetCounter = stride;

            //fill the offset
            do{
                curOffset += 1;
                dataPro[curPosition + curOffset] = 0;

                offsetCounter += 1;
            } while (offsetCounter % 4 != 0);
        }

        ptr = (System::IntPtr)dataPro;//set the data pointer to new/modified data array

        //calc the stride to nearest number which is a multiply of 4
        stride = (srcImg.size().width * srcImg.channels() + 3) & -4;

        retImg = gcnew System::Drawing::Bitmap(srcImg.size().width, srcImg.size().height,
            stride,
            System::Drawing::Imaging::PixelFormat::Format24bppRgb,
            ptr);
    }
    else{

        //no need to add a padding or recalculate the stride
        retImg = gcnew System::Drawing::Bitmap(srcImg.size().width, srcImg.size().height,
            stride,
            System::Drawing::Imaging::PixelFormat::Format24bppRgb,
            ptr);
    }
   
    array^ imageData;
    System::Drawing::Bitmap^ output;

    // Create the byte array.
    {
        System::IO::MemoryStream^ ms = gcnew System::IO::MemoryStream();
        retImg->Save(ms, System::Drawing::Imaging::ImageFormat::Png);
        imageData = ms->ToArray();
        delete ms;
    }

    // Convert back to bitmap
    {
        System::IO::MemoryStream^ ms = gcnew System::IO::MemoryStream(imageData);
        output = (System::Drawing::Bitmap^)System::Drawing::Bitmap::FromStream(ms);
    }

    return output;
}



Now we can convert cv::Mat to System.Drawing.Bitmap. You can call BitmapToMat() function to convert C# Bitmap and use converted Mat to do image processing from OpenCV, so OpenCV will produce processed image as Mat type, here you can use MatToBitmap() function to return Bitmap type object back to C#. Now you can treat this processed Bitmap as a normal Bitmap in C#.

Step 5 - Use converter functions and do image processing.

You can use above 2 functions and your own image processing code to create processed image from an input image. Here, I am trying to apply "medianBlur" opencv filter to C# Bitmap image. This is a sample code which show how to use these conversion functions along some opencv functions.


System::Drawing::Bitmap^ MyOpenCvWrapper::ApplyFilter(System::Drawing::Bitmap^ bitmap){
    Mat image = BitmapToMat(bitmap);//convert Bitmap to Mat
    if (!image.data){
        return nullptr;
    }

    Mat dstImage;//destination image

    //apply the Filter
    medianBlur(image, dstImage, 25);

    //convert Mat to Bitmap
    System::Drawing::Bitmap^ output = MatToBitmap(dstImage);

    return output;
}

Step 6 - Call from C#

Now you can call this function from C# by passing a Bitmap type object to the method and it will return the filter applied image back as a Bitmap. So you can use this returned Bitmap as a normal Bitmap in C#.

//open jpg file as Bitmap
Bitmap img = (Bitmap)Bitmap.FromFile(@"C:\Users\Public\Pictures\Sample Pictures\Tulips.jpg");

OpenCvDotNet.MyOpenCvWrapper obj = new OpenCvDotNet.MyOpenCvWrapper();
Bitmap output = obj.ApplyFilter(img);//call opencv functions and get filterred image

 output.Save("test.jpg");//save processed image

If you put this code in an event handler in C#, it will looks like this,

Step 7 - Use returned Bitmap from OpenCV in C#

you can use this returned Bitmap as a normal Bitmap in C# such as setting the image for PictureBox. Following code will open a Bitmap from a file, process in opencv to apply filter and show the results in a C# PictureBox control.

private void btnOpen_Click(object sender, EventArgs e)
{
    //allow user to open jpg file
    OpenFileDialog dlogOpen = new OpenFileDialog();
    dlogOpen.Filter = "Jpg Files|*.jpg";
    if (dlogOpen.ShowDialog() != System.Windows.Forms.DialogResult.OK)
        return;

    //open jpg file as Bitmap
    Bitmap img = (Bitmap)Bitmap.FromFile(dlogOpen.FileName);

    pbSrcImg.Image = img;//set picture box image to UI

    OpenCvDotNet.MyOpenCvWrapper processor = new OpenCvDotNet.MyOpenCvWrapper();
    Bitmap processedImg = processor.ApplyFilter(img);//call opencv functions and get filterred image

    pbDstImage.Image = processedImg;//set processed image to picture box
}

Where pbSrcImg and pbDstImage are PictureBox UI controls in C#. Once you open a jpg image it will show the UI as follow,



Download complete Visual Studio project.


From Previous Article...
So now we are going to create this wrapper for our application. Since we are dealing with .net framework, we can use CLR (Common Language Runtime) technique to create this wrapper. First you have to create a CLR project in Visual Studio. This post will describe, how to call Opencv functions from winfrom/C# and apply an Opencv filter to an image and show the Opencv window from winform.

13 comments:

  1. Thanks for your post. It's great <3<3

    ReplyDelete
    Replies
    1. Can you help me?
      Error with my project. A generic error occurred in GDI+.

      Delete
  2. I downloaded your project and received the following errors:
    The type or namespace 'OpenCvDotNet' could not be found (are you missing a using directive or an assembly reference?)

    I can't quite understand why he throws this error as I put all the projects in x64 (debug) and also added a reference to the WinForm

    ReplyDelete
    Replies
    1. Thanks for the article. I was after the conversion from Mat to .net Bitmap.
      near the end of this function, is there a need to convert the created Bitmap to memory stream and back? The most important point I found was the pixelFormat needs to be adapted in case of Grayscale (only a single channel). This function does not deal with that.

      I'm currently using this which appears to work fine:

      if (image->channels() > 1) {
      pixelFormat = System::Drawing::Imaging::PixelFormat::Format24bppRgb;
      } else {
      pixelFormat = System::Drawing::Imaging::PixelFormat::Format8bppIndexed;
      }
      bitmap = gcnew System::Drawing::Bitmap(image->cols,image->rows,image->step,pixelFormat,(IntPtr)image->data)

      Delete
  3. Hi, i've been trying to make a release on this project but am not sure where am making a mistake. When i build the release everything works fine, but when i take it to another computer it doesn't open. can you please help me? thanks

    ReplyDelete
    Replies
    1. simple solution is..... you need to copy opencv dll files from your development computer to other computer to the same directory as your release exe exists

      Delete
  4. Hello, I copy-paste your bitmap-to-mat converter, add reference to System.Drawing and compiler shows me that it "cannot convert from 'IpIImage *' to cv::Mat". What could be wrong?

    ReplyDelete
  5. Get the best google related issues faced by general user that is gmail login mail not working from one of the best technicians who have sound knowledge to provide customer service within sort period of time. Service is available for 24*7 dial 1-888-576-1584

    ReplyDelete
  6. This might be possible that your Epson laser Printer has stopped working in the ideal manner. If toner cartridges are the cause behind it, you should contact the well skilled technicians who are present at Epson Printer Helpline Number UK for best support.
    Epson Printer Helpline Number UK

    ReplyDelete
  7. If for any reason you are not able to access your Gmail account, and then dial Gmail Customer Care Number UK for assistance. We will diagnose the issue and then provide you with an appropriate solution to rectify the problem.
    Contact No. : - 0808-169-9742
    Website: - http://www.gmail-support.uk/gmail-support.php
    Face Book: - https://www.facebook.com/gmailuksupport/
    Twitter: - https://twitter.com/gmailuknumber
    Instagram: - https://www.instagram.com/i_am_rosie_taylor/
    Address: - London, United Kingdom

    ReplyDelete
  8. Hi! Thanks for the tutorial, it is very useful. Regarding the function Mat to Bitmap I'm getting an error that says: "A generic error occurred in GDI+", especifically when I try to save retImg with the empty memorystream:

    retImg->Save(ms, System::Drawing::Imaging::ImageFormat::Png);

    Any ideas? Thanks!

    ReplyDelete
    Replies
    1. I have managed to solve the problem by using different functions Bitmaptomat and mattobitmap. Pay attention that I have change AUTO_STEP and written bitmap->Stride, since if the image contains padding it won't display the image correctly. In mattobitmap I solved the problem by preconverting to Windows HBITMAP:

      Mat BitmapToMat(System::Drawing::Bitmap^ bitmap)
      {
      Mat final_mat;

      System::Drawing::Imaging::BitmapData^ bmpdata = bitmap->LockBits(System::Drawing::Rectangle(0, 0, bitmap->Width, bitmap->Height),
      System::Drawing::Imaging::ImageLockMode::ReadWrite, System::Drawing::Imaging::PixelFormat::Format24bppRgb);

      System::Drawing::Imaging::BitmapData^ bmpdata = bitmap->LockBits(System::Drawing::Rectangle(0, 0, bitmap->Width, bitmap->Height),
      System::Drawing::Imaging::ImageLockMode::ReadWrite, System::Drawing::Imaging::PixelFormat::Format32bppArgb);

      if (bitmap->PixelFormat == System::Drawing::Imaging::PixelFormat::Format8bppIndexed)
      {
      final_mat = Mat(cv::Size(bitmap->Width, bitmap->Height), CV_8UC1, bmpdata->Scan0.ToPointer(), bmpdata->Stride);
      }
      else
      {
      final_mat = Mat(cv::Size(bitmap->Width, bitmap->Height), CV_8UC3, bmpdata->Scan0.ToPointer(), cv::Mat::AUTO_STEP);
      final_mat = Mat(cv::Size(bitmap->Width, bitmap->Height), CV_8UC3, bmpdata->Scan0.ToPointer(), bmpdata->Stride);
      }

      bitmap->UnlockBits(bmpdata);

      return final_mat;
      }

      System::Drawing::Bitmap^ MatToBitmap(Mat srcImg)
      {
      Mat srcImg_;

      cvtColor(srcImg, srcImg_, COLOR_BGR2BGRA);

      HBITMAP hBit = CreateBitmap(srcImg_.cols, srcImg_.rows, 1, 32, srcImg_.data);

      System::Drawing::Bitmap^ output = System::Drawing::Bitmap::FromHbitmap((IntPtr)hBit);

      return output;
      }

      Delete