OpenCV Webcam feed not displaying in PictureBox visual studio 2015

0
votes

Hi so I used opencv to get webcam feed from my default camera and I wish to display it in a picturebox on my windows form. My webcam comes on but for some reason, the feed is never displayed on my picturebox. Please could someone help point out/solve the issue as I am stuck here right now. Thanks in advance.

In myform.h, I have this code to send the picturebox to the myform.cpp file:

System::Windows::Forms::PictureBox^ mypicbox1(void)  
{
    opencv_gui::MyForm aform;
    return aform.pictureBox1;
}

and the code to draw get the videofeed and put into my picture box in myform.cpp is:

void opencv_gui::DrawCvImage(const cv::Mat& cvImage)
{
    System::Windows::Forms::PictureBox^ pictureBox = mypicbox1();
    // only color images are supported
    assert(cvImage.type() == CV_8UC3);

    if ((pictureBox->Image == nullptr) || (pictureBox->Width != cvImage.cols) || (pictureBox->Height != cvImage.rows))
    {
        pictureBox->Width = cvImage.cols;
        pictureBox->Height = cvImage.rows;
        pictureBox->Image = gcnew System::Drawing::Bitmap(cvImage.cols, cvImage.rows);
    }

    // Create System::Drawing::Bitmap from cv::Mat
    System::Drawing::Bitmap^ bmpImage = gcnew System::Drawing::Bitmap(
        cvImage.cols, cvImage.rows, cvImage.step,
        System::Drawing::Imaging::PixelFormat::Format24bppRgb,
        System::IntPtr(cvImage.data)
    );

    // Draw Bitmap over a PictureBox
    System::Drawing::Graphics^ g = System::Drawing::Graphics::FromImage(pictureBox->Image);

    g->DrawImage(bmpImage, 0, 0, cvImage.cols, cvImage.rows);
    pictureBox->Refresh();

    delete g;
}


//camera feed
int opencv_gui::video_cap(void)
{
    VideoCapture cap;

    if (!cap.open(0)) // open the default camera (camera 0), use something different from 0 otherwise;
        return 0;
    for (;;)
    {
        Mat frame;
        cap >> frame;
        if (frame.empty()) break; // end of video stream
        DrawCvImage(frame);
        if (waitKey(10) == 27) break; // stop capturing by pressing ESC 
    }
    // the camera will be closed automatically upon exit
    // cap.close();
    return 0;
}

This is my debug log: I have used arrows like ">>>>>>>" to show important parameters and the words "RED >>>>>>>" to show error parameters.

  • cvImage { flags=1124024336 dims=2 rows=480 ...} cv::Mat&
  • allocator 0x0000000000000000 cv::MatAllocator*

    cols 640 int

  • data 0x0000026AC2146F80 unsigned char* *(*cvImage).data 0 '' unsigned char

  • dataend 0x0000026AC2227F80 unsigned char*
  • datalimit 0x0000026AC2227F80 unsigned char*
  • datastart 0x0000026AC2146F80 unsigned char* dims 2 int flags 1124024336 int rows 480 int

  • size { p=0x000000DBDFF0DE80 } cv::MatSize

  • p 0x000000DBDFF0DE80 int* *(*cvImage).size.p 0 int

  • step { p=0x000000DBDFF0DEC8 buf={Length=2} } cv::MatStep
  • buf {Length=2} unsigned __int64[]
  • p 0x000000DBDFF0DEC8 unsigned __int64*
  • u 0x0000026AA4814780 cv::UMatData*
  • RED >>>>>>> bmpImage 0x0000026aa62a1fd0 { defaultTransparentColor= } System::Drawing::Bitmap^
  • RED >>>>>>> System::Drawing::Image^ 0x0000026aa62a1fd0 { nativeImage=2657511292512 rawData= userData= } System::Drawing::Image^ defaultTransparentColor System::Drawing::Color g System::Drawing::Graphics^
  • RED >>>>>>> pictureBox 0x0000026aa62a1388 { borderStyle=System::Windows::Forms::BorderStyle::None image=0x0000026aa62a1fa0 sizeMode=System::Windows::Forms::PictureBoxSizeMode::Normal ...} System::Windows::Forms::PictureBox^
  • RED >>>>>>> System::Windows::Forms::Control^ 0x0000026aa62a1388 { ControlKeyboardRouting= PaletteTracing= FocusTracing= ...} System::Windows::Forms::Control^ AllowDrop false bool BorderStyle System::Windows::Forms::BorderStyle::None System::Windows::Forms::BorderStyle CausesValidation true bool
  • CreateParams 0x0000026aa62a15d8 { className= caption= style=1442840576 ...} System::Windows::Forms::CreateParams^ DefaultImeMode System::Windows::Forms::ImeMode::Disable System::Windows::Forms::ImeMode
  • DefaultSize {Width=100 Height=50} System::Drawing::Size EVENT_SIZEMODECHANGED 0x0000026aa629af50 System::Object^
  • ErrorImage 0x0000026aa629b110 { nativeImage=2657511192608 rawData= userData= } System::Drawing::Image^
  • Font 0x0000026aa6280148 {} System::Drawing::Font^
  • ForeColor {RGB=0x0} System::Drawing::Color
  • Image 0x0000026aa62a1fa0 { nativeImage=2657511290992 rawData= userData= } System::Drawing::Image^
  • [System::Drawing::Bitmap^] 0x0000026aa62a1fa0 { defaultTransparentColor= } System::Drawing::Bitmap^
  • System::MarshalByRefObject^ 0x0000026aa62a1fa0 { __identity= } System::MarshalByRefObject^ Flags 2 int
  • FrameDimensionsList {Length=1} array ^ Height 480 int HorizontalResolution 96.000000 float
  • Palette 0x0000026aa62a25e8 { flags=1129621568 entries={Length=0} } System::Drawing::Imaging::ColorPalette^
  • PhysicalDimension { width=640.00000 height=480.00000 } System::Drawing::SizeF PixelFormat System::Drawing::Imaging::PixelFormat::Format32bppArgb System::Drawing::Imaging::PixelFormat PropertyIdList {Length=0} array ^ PropertyItems {Length=0} array ^
  • RawFormat 0x0000026aa62a3488 { ...} System::Drawing::Imaging::ImageFormat^
  • Size {Width=640 Height=480} System::Drawing::Size Tag System::Object^ VerticalResolution 96.000000 float Width 640 int nativeImage 2657511290992 __int64 rawData array ^ userData System::Object^ ImageLocation System::String^
  • ImageRectangle {X=0 Y=0 Width=640 Height=480} System::Drawing::Rectangle ImeMode System::Windows::Forms::ImeMode::Disable System::Windows::Forms::ImeMode
  • InitialImage 0x0000026aa629c568 { nativeImage=2657511196256 rawData= userData= } System::Drawing::Image^ PICTUREBOXSTATE_asyncOperationInProgress 1 int PICTUREBOXSTATE_cancellationPending 2 int PICTUREBOXSTATE_inInitialization 64 int PICTUREBOXSTATE_needToLoadImageLocation 32 int PICTUREBOXSTATE_useDefaultErrorImage 8 int PICTUREBOXSTATE_useDefaultInitialImage 4 int PICTUREBOXSTATE_waitOnLoad 16 int RightToLeft System::Windows::Forms::RightToLeft::No System::Windows::Forms::RightToLeft SizeMode System::Windows::Forms::PictureBoxSizeMode::Normal System::Windows::Forms::PictureBoxSizeMode TabIndex 1 int TabStop false bool Text "" System::String^ WaitOnLoad false bool borderStyle System::Windows::Forms::BorderStyle::None System::Windows::Forms::BorderStyle contentLength 0 int currentAsyncLoadOperation System::ComponentModel::AsyncOperation^ currentlyAnimating false bool
  • defaultErrorImage 0x0000026aa629b110 { nativeImage=2657511192608 rawData= userData= } System::Drawing::Image^
  • defaultErrorImageForThread 0x0000026aa629b110 { nativeImage=2657511192608 rawData= userData= } System::Drawing::Image^ defaultErrorImageKey 0x0000026aa629af08 System::Object^
  • defaultInitialImage 0x0000026aa629c568 { nativeImage=2657511196256 rawData= userData= } System::Drawing::Image^
  • defaultInitialImageForThread 0x0000026aa629c568 { nativeImage=2657511196256 rawData= userData= } System::Drawing::Image^ defaultInitialImageKey 0x0000026aa629aef0 System::Object^
  • errorImage 0x0000026aa629b110 { nativeImage=2657511192608 rawData= userData= } System::Drawing::Image^ handleValid false bool
  • image 0x0000026aa62a1fa0 { nativeImage=2657511290992 rawData= userData= } System::Drawing::Image^ imageInstallationType ImageInstallationType::DirectlySpecified ImageInstallationType imageLocation System::String^
  • initialImage 0x0000026aa629c568 { nativeImage=2657511196256 rawData= userData= } System::Drawing::Image^
  • [System::Drawing::Bitmap^] 0x0000026aa629c568 { defaultTransparentColor= } System::Drawing::Bitmap^
  • System::MarshalByRefObject^ 0x0000026aa629c568 { __identity= } System::MarshalByRefObject^ Flags 73744 int
  • FrameDimensionsList {Length=1} array ^ Height 16 int HorizontalResolution 96.000000 float
  • RED >>>>>>>Palette 0x0000026aa629ea60 { flags=951278840 entries={Length=0} } System::Drawing::Imaging::ColorPalette^
  • PhysicalDimension { width=14.000000 height=16.000000 } System::Drawing::SizeF PixelFormat System::Drawing::Imaging::PixelFormat::Format24bppRgb System::Drawing::Imaging::PixelFormat PropertyIdList {Length=0} array ^ PropertyItems {Length=0} array ^
  • RED >>>>>>> RawFormat 0x0000026aa629f900 { ...} System::Drawing::Imaging::ImageFormat^
  • Size {Width=14 Height=16} System::Drawing::Size Tag System::Object^ VerticalResolution 96.000000 float Width 14 int nativeImage 2657511196256 __int64 rawData array ^ userData System::Object^ RED >>>>>>> internalSyncObject 0x0000026aa629d900 System::Object^ loadCompletedDelegate System::Threading::SendOrPostCallback^ loadCompletedKey 0x0000026aa629af20 System::Object^ loadProgressChangedKey 0x0000026aa629af38 System::Object^ loadProgressDelegate System::Threading::SendOrPostCallback^ localImageStreamReader System::IO::StreamReader^
  • pictureBoxState { data=12 } System::Collections::Specialized::BitVector32 readBlockSize 4096 int readBuffer array ^
  • savedSize {Width=640 Height=480} System::Drawing::Size sizeMode System::Windows::Forms::PictureBoxSizeMode::Normal System::Windows::Forms::PictureBoxSizeMode tempDownloadStream System::IO::MemoryStream^ totalBytesRead 0 int uriImageStream System::IO::Stream^
2
c++winformsvisual-studioopencvvideo-capture
Since I don't see any synchronization, I assume opencv_gui::video_cap runs on the main GUI thread. As such, seeing an infinite loop there is already a sign of potential trouble. While waitKey might kinda-sorta work in, I wouldn't rely on that for anything other than the OpenCV UI. This is a pretty awkward design in a GUI app anyway. You already wait between frames, so why not use a timer? – Dan Mašek
Thanks Dan for the comment, I intend to clean up the code soon regarding the loop but removing the loop and capturing manually still doesn't display frames on the picture box which is my main issue. – Kosy Onyenso
Can you try to step through that function in the debugger? What are the values of cvImage.cols, cvImage.rows, cvImage.step ? Does cvImage.data contain some reasonable values? BTW, OpenCV images are BGR, and you interpret the data as RGB. You probably need a cvtColor there to switch it around. – Dan Mašek
Hi Dan, I have put a copy of my debug log into the question. The image size and data parameters are zero but the other parameters have reasonable value(640 and 480)...I have also signified which of the debug lines were highlighted in red for you to look at. Thanks for the help so far. – Kosy Onyenso

2 Answers

0
votes

Had to edit my code a bit but I pretty much solved my issue using the answer provided here: https://stackoverflow.com/a/12628861/5728859

0
votes

try this

private: System::Void button2_MouseClick(System::Object^  sender, System::Windows::Forms::MouseEventArgs^  e)
{
    if (button2->Text == "Stop")
    {
        button2->Text = "Start";

    }
    else if (button2->Text == "Start")
    {
        button2->Text = "Stop";

    }

     VideoCapture capture(0);
     Mat frame;

     while (button2->Text == "Stop")
     {
         capture.read(frame);
         System::Drawing::Graphics^ graphics2 = pictureBox1->CreateGraphics();
         System::IntPtr ptr2(frame.ptr());
         System::Drawing::Bitmap^ b2 = gcnew System::Drawing::Bitmap(frame.cols,
             frame.rows, frame.step, System::Drawing::Imaging::PixelFormat::Format24bppRgb, ptr2);
         System::Drawing::RectangleF rect2(0, 0, pictureBox1->Width, pictureBox1->Height);
         graphics2->DrawImage(b2, rect2);

     }
 }