Find the differences between two noisy, lighting variant images (histogram alignment and more)

Question

I am trying to extract a single object from a picture taken by a camera. A picture of the camera's view without the object exists. The camera's focus does not change, but the lighting does when an object is in view. The object is solid but not of constant shade of gray. Here is a picture of the reference image on the right and another image with an object on the left. The pictures are monochrome.

All the processing is happening after the image acquisition and computation time is not a big issue. The precision of the algorithm is more important.

Most questions I have found here deal with the question if two pictures are similar or not, but I am interested in finding the region that the object occupies in the image for subsequent measurements.

So far, I have tried different combinations of subtracting and then binarizing the blurred images with a threshold, but this is not lighting invariant. Multiplying the reference image beforehand by the relative lighting difference does not yield satisfactory results either. Maybe it could work if a better way to simulate a different exposure is used.

I have also tried subtracting the LoG filtered images and a few crude neighborhood pixel comparisons without success.

It feels like a very intuitive task that some kind of descriptor comparison should be able to handle, but I am struggling to find a good solution. Does anyone know of a good method that I've been missing?

Thanks in advance.

Thanks to Franco Callari's answer, I stumbled upon histogram matching which to my surprise wasn't covered by ready-made OpenCV functions. Since this seems like a common problem, I might as well post my inefficient hack-up here in case someone wants to use it.

// Aligns the histogram of the source image to match that of the target image. 
// Both are evaluated in a ROI, not across the whole picture.
// The function assumes 8-bit grayscale images.
static void alignHistogram(Mat alignsource, const Mat& aligntarget, 
    const int rowstart = 500, const int rowend = 700, 
    const int colstart = 0, const int colend = 1000)    
{   
    // 1) Calculate target and source histogram in region of interest
    // 2) Compute the integral of each histogram (cumulative distribution function)
    // 3) Set brightness of each pixel in the source image to the brightness of the target where the CDF values are equal

    Mat ROIsource = alignsource(Range(rowstart, rowend), Range(colstart, colend));
    Mat ROItarget = aligntarget(Range(rowstart, rowend), Range(colstart, colend));
    MatND hist_source, hist_target;

    int bins = 256, int histSize[] = {bins};
    float sranges[] = { 0, 256 };        const float* ranges[] = { sranges };
    int channels[] = {0};
    calcHist( &ROItarget, 1, channels, Mat(), hist_target, 1, histSize, ranges, true, false );
    calcHist( &ROItarget, 1, channels, Mat(), hist_source, 1, histSize, ranges, true, false );

    Mat CDF_target_2d, CDF_source_2d; 
    integral(hist_target, CDF_target_2d);
    integral(hist_source, CDF_source_2d);
    Mat CDF_target = CDF_target_2d.col(1),  CDF_source = CDF_source_2d.col(1);

    // Cycle through source image inefficiently and adjust brightness
    for (int i = 0; i < alignsource.rows; i++)
    {
        for (int j = 0; j < alignsource.cols; j++)
        {
            int source_brightness = alignsource.at<unsigned char>(i, j);
            double cdf_source_value = CDF_source.at<double>(source_brightness);

            int target_brightness = 0;
            while (target_brightness < 256) {
                if (CDF_target.at<double>(target_brightness) > cdf_source_value)
                    break;
                target_brightness++;
            }
            alignsource.at<unsigned char>(i, j) = target_brightness;
        }
    }
}

Adjusting the lighting helps to get a better first guess of the object, but it"s not enough to get a precise contour, especially when the background is not very different from the object or rich in features. That's as far as I got at the moment.

Francesco Callari Francesco Callari · Accepted Answer · 2013-03-28T14:13:34

If the the camera is not moved nor the background changed (as it appears from the sample photo), the difference in global illumination is likely due (for the most part) to either of two factors - camera with auto-exposure making a different choice of f/stop or exposure time when the subject is in the scene, or a time-varying light source within the exposure time window (e.g. 60Hz line hum in the lamp). The latter is to be excluded if the illuminator is a strobe (and enough time between shots is given for the strobe to recharge).

I say "for the most part" above because, with the subject taking up a large portion of the frame, light reflected from it does affect global illumination as well, but in your case this is likely a second order effect.

Your best approach is likely to be to better control the capture - at a minimum, disable auto-exposure in the camera, use a ballasted light (if it's not a strobe).

If you cannot (or in addition), you should start with global histogram alignment, rather than equalization. Global histogram equalization, as suggested by other posters, is likely to hurt, because the subject's pixel values will be part of the histogram, not just the background. However, if the camera is not moved, you can pre-identify in the image frame regions that are known to be background, and sample the histogram only from them in both the "background only" and "with subject" images. You can then find the values at, say, the top and bottom 5% of the dynamic range, and just apply a global scaling to they match.

Find the differences between two noisy, lighting variant images (histogram alignment and more)

5 Answers