Rectangle detection on game board with opencv python

Question

I'm new to opencv using python, so please bear with me. I have a tray with different sizes of circles like in the link. https://is.alicdn.com/img/pb/810/421/429/429421810_364.jpg This is not the actual image I have, but it is very similar to the one above. I have to detect the tray in the image and find the contours for all the holes (circles) in the tray. The tray might be tilted depending on the user taking the image. So far, I used a Gaussian blur and canny edge detection on the image, and closed the gaps in the canny edge detection. This is the image after Canny Edge Detection This is the image after morphology. Then I used findcontours and tried to find the largest contour with 4 vertices, which would ideally be the tray itself. The contour detection is only able to identify the left vertical border and top horizontal border. It is not able to recognize the 4 edges of the tray.

This is my code so far:

import numpy as np
import cv2
import matplotlib.pyplot as plt

image = cv2.imread("img.jpg")
image = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
image = cv2.GaussianBlur(image, (3, 3), 0)
image_canny = cv2.Canny(image, 30, 200)
kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (7, 7))
gaps_closed = cv2.morphologyEx(image_canny, cv2.MORPH_CLOSE, kernel)
_, contours, _= cv2.findContours(gaps_closed.copy(), cv2.RETR_TREE, 
cv2.CHAIN_APPROX_SIMPLE)
contours = sorted(contours, key = cv2.contourArea, reverse = True)
[:5]
Cnt = None
for c in contours:
    peri = cv2.arcLength(c, True)
    approx = cv2.approxPolyDP(c, 0.04 * peri, True)

    if len(approx) == 4:
        Cnt = approx
        break

cv2.drawContours(image, [Cnt], -1, (0, 255, 0), 4)
plt.imshow(image)
plt.show()

Answer 1

Tuning the parameters of cv2.canny should make it easier to detect the edges of the tray. However, using cv2.HoughLines might be a better way to detect the tray, since HoughLines does not requite a complete edge to be detected.

Answer 2

Performing a histogram equalization on the input should improve the contrast in your image. This will improve the edge detection in the image.

Since the holes in your image seem small, morphological operations may not be the way to go. That'll blot out the holes and you may not be able to retrieve them back.

Opencv has implementations for the Hough circle transform. Here's a python tutorial to an example: http://docs.opencv.org/3.0-beta/doc/py_tutorials/py_imgproc/py_houghcircles/py_houghcircles.html. Using this should solve your problems.

If this doesn't work, posting your actual input image would be helpful