How does Histogram Backprojection in numpy work?

Question

I'me new to this platform and also new to computer vision world. I'm working on a project where i'm using histogram BackProjection to detect a colored object. There are two ways to implement this method, either using numpy or a built-in function form opencv (cv2.calcBackProject). However, I do really want to use the numpy version since it allows to better understand what happens. The code for numpy implementation is shown below:

1 import cv2
2 import numpy as np
3 import matplotlib.pyplot as plt
4 #roi is the object or region of object we need to find
5 roi = cv2.imread('D:/downloads/messi_ground.jpg')
6 hsv = cv2.cvtColor(roi,cv2.COLOR_BGR2HSV)
7 #target is the image we search in
8 target = cv2.imread('D:/downloads/messi.jpg')
9 hsvt = cv2.cvtColor(target,cv2.COLOR_BGR2HSV)
10 # Find the histograms using calcHist.
11 M = cv2.calcHist([hsv],[0, 1], None, [180, 256], [0, 180, 0, 256] )
12 I = cv2.calcHist([hsvt],[0, 1], None, [180, 256], [0, 180, 0, 256] )

13 R = M/I
14 h,s,v = cv2.split(hsvt)

15 B = R[h.ravel(),s.ravel()
16 B = np.minimum(B,1)
17 B = B.reshape(hsvt.shape[:2])

This algorithm finds pixels in the target image which are similar to the ones in the Model histogram (in my case the histogram of the roi, M)

So my questions are:

what literally happens when one histogram is divided over another (line 13)?
how the resulted histogram (in this case R in line 13) is back projected using the h,s channel of the target image? In other words, how does line 15 back project the Resulted R-histogram?
Line 16 supposes to make numbers between 0 and 1 in order to view pixel values as probability belonging to the Model histogram. However, I skipped it and got the same results though, so why the algorithms uses it ?

Unfortunately, the paper that explains this method is very abstract and doesn't explicit the functionality of the algorithm (Paper: Indexing via color histogram) I know that this websites supposes to help with direct programming problems. However, I think it should also helps understanding some operations related to programming.

Abhi25t Abhi25t · Accepted Answer · 2020-08-07T14:27:15

In the paper- Indexing via color histograms, authors Michael J. Swain and Dana H. Ballard focus on - What and Where are the colors in the target object image?

First we calculate the histogram model of a feature and then use it to find this feature in target image.

Line 11-12 are calculating 2-D histograms which are different from normal ones. A 2-D histogram counts the occurrence of combinations of intensities.

We want to find similarities in I and M. In Line 13 when we divide M by I, the resultant matrix (image) R elements (pixels) will be non-zero only when both M and I elements (pixels) are non-zero. Else, they will be zero or Nan. These non-zero pixels region could have our target object. Now we find the pixels in target image corresponding to these non-zero pixels, using the hue (H) & saturation (S) values of target image pixels (extracted in Line 14)

Line 15 is filtering out these non-zero values using h & s. The Nan values are simply rejected and zero values are black anyway. Its just a filtering operation similar to thresholding - that's why uses flattened array by ravel.

Skipping Line 16 might have worked in your case perhaps because your roi would contain one of the R/G/B colors whose value is close to high end of scale i.e. near 1 on 0-1 scale and near 255 in 0-255 scale. So dividing a smaller number by this big number keeps it already <1

How does Histogram Backprojection in numpy work?

1 Answers