java opencv png image with alpha channel over video issue

Question

everybody. I'm trying to place ".png" image over the video capture using java and opencv.

I performed the tutorial from the official opencv java website http://opencv-java-tutorials.readthedocs.io/en/latest/04-opencv-basics.html

In this tutorial everything works fine with png image from the tutorial. But when i try to load my own png image with alpha channel, i get the wrong result. Png image displays with white spaces.

Here is the print screens of video capture with both images:

correct output

incorrect output

my png image

Here is the code of loading image:

@FXML
protected void loadLogo() {
    if(logoCheckBox.isSelected()) {
        this.logo = Imgcodecs.imread("resources/Poli.png", Imgcodecs.IMREAD_COLOR);
    }
}

Here is the code of video capture and placing png image over it:

Mat frame = new Mat();

// read the current frame
this.capture.read(frame);
Rect roi = new Rect(frame.cols() - logo.cols(), frame.rows() - logo.rows(), logo.cols(), logo.rows());
Mat imageROI = frame.submat(roi);

Core.addWeighted(imageROI, 1.0, logo, 1.0f, 0.0, imageROI);

And here is how output image place inside ImageView:

MatOfByte buffer = new MatOfByte();
Imgcodecs.imencode(".png", frame, buffer);
Image imageToShow = new Image(new ByteArrayInputStream(buffer.toArray()));
Image imageToShow = Utils.mat2Image(frame);
currentFrame.setImage(imageToShow);

I know that parameter Imgcodecs.IMREAD_COLOR load image with 3 channels, without alpha, but when i change this parameter to IMREAD_UNCHANGED i got this error:

OpenCV Error: Sizes of input arguments do not match (The operation is neither 'array op array' (where arrays have the same size and the same number of channels)

This is appear becouse the input Mat with video capture contains only 3 channels.

My question is: how can i load and place png image over video capture correctly ?

P.S When i fill the background of png image with black, its displayd correct, but image is still stay transparent.

Часть Пустоты Часть Пустоты · Accepted Answer · 2017-11-13T11:24:33

The answer is that RGBA image cannot be placed over RGB video capture. Placing only RBG components of png image gives "wrong result".

For now, the best solution i found is to load png image in RBGA mode COLOR_UNCHANCHED wiht 4 channels, take the 4-th alpha channel and use it as mask for displaying image with function "copyTo" instead of "addWeighted".

But this method doesnt work with translucent parts of PNG image like shadows, and objects with low opacity.

Here is the code of loading image with separate RGB and alpha channel:

Mat logo = new Mat();
Mat logo_alpha = new Mat();
Mat logo_with_alpha = Imgcodecs.imread("resources/TREE_1.png", Imgcodecs.IMREAD_UNCHANGED);         
Vector<Mat> rgba = new Vector<Mat>();
// split RBGA image for separate channels
Core.split(logo_with_alpha, rgba);
// get alpha channel
logo_alpha = rgba.get(3);
// remove alpha channel to get RGB image
rgba.remove(rgba.size()-1);
// get image with only RGB components
Core.merge(rgba, this.logo);

Here is the code of video capturing and placing png image over it:

Mat frame = new Mat();
// read the current frame
this.capture.read(frame);
Rect roi = new Rect(frame.cols() - logo.cols(), frame.rows() - logo.rows(), logo.cols(), logo.rows());
Mat imageROI = frame.submat(roi);
// place logo with alpha on the frame
logo.copyTo(imageROI, logo_alpha);

java opencv png image with alpha channel over video issue

1 Answers