how to prevent or deal with byte overflow and byte underflow when manipulate individual pixel in Opencv 3.1 Java

Question

I am trying to manipulate individual image pixel with OpenCV 3.1 Java. And OpenCV read an Image as byte array. I want to convert color image to gray image without changing the total of the image channels with function Y = R*0.299 + G*0.587 + B*0.114, so the result will still be an RGB image (3 color channels). From my understanding, because the the sum of 0.299 + 0.587 + 0.114 = 1, and although the byte range from -128 to 127, there should be no problem with underflow or overflow, when casting to byte. But the result is weird, because some white area of the image become black, and some black area become white. I assume that underflow or overflow happened. Here is my code:

List<Mat> channels = new ArrayList<>();
Core.split(intImage, channels);
int totalInt = (int)(channels.get(0).total());
byte[] blue = new byte[totalInt];
byte[] green = new byte[totalInt];
byte[] red = new byte[totalInt];
channels.get(0).get(0, 0, blue);
channels.get(1).get(0, 0, green);
channels.get(2).get(0, 0, red);
for (int i = 0; i < totalInt; i++) {
    byte s = (byte)(blue[i]*0.114 + green[i]*0.587 + red[i]*0.299);
    blue[i] = red[i] = green[i] = s;
}
channels.get(0).put(0, 0, blue);
channels.get(1).put(0, 0, green);
channels.get(2).put(0, 0, red);
Mat gray = new Mat();
Core.merge(channels, gray);

I have try to convert the image to CvType.CV_16S, which represent the unsigned short, and it worked without problem so far . Convert to CV_8UC3 still in byte. I am worrying about the heap problem, because when i try with int, which is CV_32S, heap error occur with some large image. So here is my question:

If could, how can I prevent or deal with those overflow/underflow. I am still consider using byte because it will reduce the heap/memory usage.
If the first is the only option, how can I can convert CV_16S directly to BufferedImage without convert back to the original Mat type, because I am using Swing to display the image.

I found the method to convert from Mat to BufferedImage as follows:

public BufferedImage toBufferedImage(Mat matrix) {
    int type = BufferedImage.TYPE_BYTE_GRAY;
    if (matrix.channels() > 1) {
        type = BufferedImage.TYPE_3BYTE_BGR;
    }
    BufferedImage image = new BufferedImage(matrix.cols(), 
            matrix.rows(), type);
    final byte[] targetPixels = 
            ((DataBufferByte)image.getRaster().getDataBuffer()).getData();
    matrix.get(0, 0, targetPixels);
    return image;
}

Please explain, what's happening with the byte casting.

Marco13 Marco13 · Accepted Answer · 2016-12-13T10:47:55

The computation in

blue[i]*0.114 + green[i]*0.587 + red[i]*0.299

(which you should definitely change to

red[i]*0.299 + green[i]*0.587 + blue[i]*0.114

to keep the RGB order intact!)

is taking place with the values of the bytes. And as you already noted, these values are negative. This causes wrong results to be computed.

Although for a (signed) byte, the nagative value -16 and the positive value 240 are represented with the same bit pattern, they are still different values, and this has to be taken into account during the computation.

Consider the following example:

public class PixelBytes
{
    public static void main(String[] args)
    {
        byte r = (byte)(100);
        byte g = (byte)(240);
        byte b = (byte)0;
        compute(r, g, b);
    }

    private static byte compute(byte r, byte g, byte b)
    {
        byte sr = (byte) (r * 0.299);
        byte sg = (byte) (g * 0.587);
        byte sb = (byte) (b * 0.114);
        byte s = (byte) (sr + sg + sb);

        byte tr = (byte) (toUnsignedInt(r) * 0.299);
        byte tg = (byte) (toUnsignedInt(g) * 0.587);
        byte tb = (byte) (toUnsignedInt(b) * 0.114);
        byte t = (byte) (tr + tg + tb);

        System.out.println("For " + r + " " + g + " " + b);
        System.out.println("add " + sr + " " + sg + " " + sb + " to get " + s);
        System.out.println("or  " + tr + " " + tg + " " + tb + " tp get " + t);

        return s;
    }

    // Same as Byte#toUnsignedInt in Java 8
    private static int toUnsignedInt(byte b)
    {
        return ((int) b) & 0xff;        
    }
}

The output will be

For 100 -16 0
add 29 -9 0 to get 20
or  29 -116 0 tp get -87

clearly showing that the results differ, depending on whether the negative values are used, or whether the values are converted into their "real" unsigned values first.

So the computation of the gray pixel value could be done with a method like this:

private static byte computeLuminance(byte r, byte g, byte b)
{
    int ir = r & 0xFF;
    int ig = g & 0xFF;
    int ib = b & 0xFF;
    return (byte)(ir * 0.299 + ig * 0.587 + ib * 0.114);
}

how to prevent or deal with byte overflow and byte underflow when manipulate individual pixel in Opencv 3.1 Java

1 Answers