OpenCV detecting a single symbol with multiple bounding boxes

Question

I am using OpenCV to put bounding boxes on handwritten math equation inputs. Currently, my code sometimes places multiple smaller bounding boxes around different parts of a singular image instead of creating one large box around the image. I'm not sure why this is happening. My current code to filter the image and find the contours to draw the bounding box is as follows:

    img = cv2.imread(imgpath)

    morph = img.copy()
    morph = cv2.fastNlMeansDenoising(img)

    kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (1, 1))
    morph = cv2.morphologyEx(morph, cv2.MORPH_CLOSE, kernel)
    morph = cv2.morphologyEx(morph, cv2.MORPH_OPEN, kernel)

    kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (3, 15))

    # take morphological gradient
    gradient_image = cv2.morphologyEx(morph, cv2.MORPH_GRADIENT, kernel)

    gray = cv2.cvtColor(gradient_image, cv2.COLOR_BGR2GRAY)

    img_grey = cv2.morphologyEx(gray, cv2.MORPH_CLOSE, kernel)

    blur = cv2.medianBlur(img_grey,3)


    ret, thing = cv2.threshold(blur, 0.0, 255.0, cv2.THRESH_BINARY + cv2.THRESH_OTSU)

    img_dilation = cv2.dilate(thing, kernel, iterations=3)


    conturs_lst = cv2.findContours(img_dilation, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)[-2]

An example of the actual result is as follows:

OG Image:

nathancy nathancy · Accepted Answer · 2019-05-13T05:20:41

You have the right idea but I think you're overusing cv2.morphologyEx to continuously erode and dilate the image. You mention your problem:

Currently, my code sometimes places multiple smaller bounding boxes around different parts of a singular image instead of creating one large box around the image.

When you use cv2.findContours, its working correctly but since your contours are actually blobs instead of one interconnected singular image, it creates multiple bounding boxes. To remedy this problem, you can dilate the image to connect the blobs together.

I've rewrote your code without the extra cv2.morphologyEx repetitions. The main idea is as follows:

Convert the image into grayscale
Blur image
Threshold image to separate background from desired object
Dilate image to connect blobs to form a singular image
Find contours and filter contours using threshold min/max area

Threshold image to isolate desired sections. Note some of the contours have broken connections. To fix this, we dilate the image to connect the blobs.

Dilate image to form singular objects. Now note we have the unwanted horizontal section at the bottom, we can find contours and then filter using area to remove that section.

Results

import numpy as np
import cv2

original_image = cv2.imread("1.jpg")
image = original_image.copy()

gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
blurred = cv2.GaussianBlur(gray, (3, 3), 0)
thresh = cv2.threshold(blurred, 160, 255, cv2.THRESH_BINARY_INV)[1]
kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (3,3))
dilate = cv2.dilate(thresh, kernel , iterations=4)

cv2.imshow("thresh", thresh)
cv2.imshow("dilate", dilate)

# Find contours in the image
cnts = cv2.findContours(dilate.copy(), cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE)
cnts = cnts[0] if len(cnts) == 2 else cnts[1]

contours = []

threshold_min_area = 400
threshold_max_area = 3000

for c in cnts:
    x,y,w,h = cv2.boundingRect(c)
    area = cv2.contourArea(c)
    if area > threshold_min_area and area < threshold_max_area:
        # cv2.drawContours(original_image,[c], 0, (0,255,0), 3)
        cv2.rectangle(original_image, (x,y), (x+w, y+h), (0,255,0),1)
        contours.append(c)

cv2.imshow("detected", original_image) 
print('contours detected: {}'.format(len(contours)))
cv2.waitKey(0)

OpenCV detecting a single symbol with multiple bounding boxes

1 Answers