groupRectangles on OpenCV not performing as expected

Question

So I'm trying to detect axis labels from a graph automatically using OpenCV. Now, my program detects the labels relatively well and outputs a list of rectangles, but one axis label value is usually separated into many rectangles. I want to group these together, and I heard cv2.groupRectangles() can get it done.

So far, I can detect the following boxes:

array([[[ 76, 620, 107, 635]],
       [[ 87, 540,  96, 554]],
       [[ 77, 459, 100, 473]],
       [[ 77, 377, 107, 392]],
       [[ 77, 297, 100, 311]],
       [[ 68, 217, 101, 231]],
       [[ 86, 139,  94, 147]],
       [[ 68, 135, 107, 150]],
       [[ 69,  54, 107,  69]],
       [[ 77,  54,  97,  69]],
       [[545, 641, 580, 655]],
       [[454, 640, 489, 655]],
       [[364, 641, 399, 655]],
       [[375, 641, 399, 655]],
       [[364, 640, 373, 654]],
       [[636, 638, 671, 655]],
       [[647, 641, 671, 655]],
       [[660, 643, 667, 651]],
       [[273, 638, 309, 655]],
       [[273, 641, 284, 655]],
       [[284, 640, 309, 655]],
       [[183, 638, 214, 655]],
       [[183, 641, 196, 655]],
       [[105, 637, 115, 655]],
       [[ 77, 625, 102, 634]]])

As shown in the graph:graph with all boxes

I use groupRectangles as below:

def groupRect(rectarray, rectthreshold=1, eps=0.1):
    """
    Groups similar rectangles together in rectangle array \n
    Input : Rectangle Array \n
    Output : Rectangle Array without split parts
    """
    results = cv.groupRectangles(np.concatenate((rectarray, rectarray)),1,eps=eps)[0]
    results = [[group] for group in results]
    return np.array(results)

I repeat the rectarray so that the algorithm keeps any rectangles that do not overlap with any other rectangle. However, as you can see from the resulting graph below, there are a few problems:

1) It gets rid of rectangle around 0.01

2) It doesn't get rid of rectangle at 125 where one is totally enclosed by the other

If anyone could help me understand why the groupRectangles code is giving this output, or suggest another algorithm to do this, I would be really grateful.

Also, is it possible to take the biggest enclosed area of the overlapping rectangles rather than taking the average of two rectangles?

Graph without boxes

Graph with boxes after groupRectangles is used

Yeah, I have. When it's 0, no change occurs (since as long as there is 1 rectangle it does not do anything). When it's 2, only rectangles which have overlaps are kept, the rest are removed) — hl teoh

Jeru Luke Jeru Luke · Accepted Answer · 2018-07-13T14:25:05

I modified the eps parameter to 0.05.

results = cv2.groupRectangles(np.concatenate((rect_old, rect_old)),1,eps=0.05)[0]
img2 = img.copy()
for r in results:
    cv2.rectangle(img2,(r[0], r[1]),(r[2], r[3]),(255,0,0),1)

EDIT

Here is some info on the parameters take more this page:

Parameters:

rectList – Input/output vector of rectangles. Output vector includes retained and grouped rectangles. (The Python list is not modified in place.)

groupThreshold – Minimum possible number of rectangles minus 1. The threshold is used in a group of rectangles to retain it.

eps – Relative difference between sides of the rectangles to merge them into a group.

groupRectangles on OpenCV not performing as expected

1 Answers