How to detect parking signs using OpenCV?

Question

Please note, I am a complete beginner in computer vision and OpenCV(Java).

My objective is to identify parking signs, and to draw bounding boxes around them. My problem is that the four signs from the top (with red borders) were not identified (see last image). I am also noticing that the Canny edge detection does not capture the edges of these four signs (see second image). I have tried with other images, and got the same results. My approach is as follows:

Load the image and convert it to gray scale
Pre-process the image by applying bilateralFilter and Gaussian blur
Execute Canny edge detection
Find all contours
Calculate the perimeter with arcLength and approximate the contour with approxPolyDP
If approximated figure has 4 points, then assuming it is a rectangle hence adding the contour

Finally, draw the contours that has 4 points exactly.

    Mat filtered = new Mat();

    Mat edges = new Mat(src.size(), CvType.CV_8UC1);
    Imgproc.cvtColor(src, edges, Imgproc.COLOR_RGB2GRAY);

    Imgproc.bilateralFilter(edges, filtered, 11, 17, 17);

    org.opencv.core.Size s = new Size(5, 5);
    Imgproc.GaussianBlur(filtered, filtered, s, 0);

    Imgproc.Canny(filtered, filtered, 170, 200);

    List<MatOfPoint> contours = new ArrayList<MatOfPoint>();
    Imgproc.findContours(filtered, contours, new Mat(), Imgproc.RETR_LIST, Imgproc.CHAIN_APPROX_SIMPLE);

    List<MatOfPoint> rectangleContours = new ArrayList<MatOfPoint>();
    for (MatOfPoint contour : contours) {
        MatOfPoint2f dst = new MatOfPoint2f();
        contour.convertTo(dst, CvType.CV_32F);
        perimeter = Imgproc.arcLength(dst, true);
        approximationAccuracy = 0.02 * perimeter;
        MatOfPoint2f approx = new MatOfPoint2f();

        Imgproc.approxPolyDP(dst, approx, approximationAccuracy, true);
        if (approx.total() == 4) {
            rectangleContours.add(contour);
            Toast.makeText(reactContext.getApplicationContext(), "Rectangle detected" + approx.total(), Toast.LENGTH_SHORT).show();
        }
    }

    Imgproc.drawContours(src, rectangleContours, -1, new Scalar(0, 255, 0), 5);

Very happy to get advice on how I could resolve this issue, even if it implies changing my stratergy.

This is a problem where you can use template matching given you have a known set of parking signs. Check this tutorial out opencv-python-tutroals.readthedocs.io/en/latest/py_tutorials/… — Knight Forked
Thanks for the quick response. Correct me if im wrong, template matching seems to work only for known parking signs (like the truck sign above). However, there are signs containing some area specific rules in text format (e.g. 6-11 in image above). It is impossible to capture all text variations. Any suggestions? Thanks. — John
I would give a shot using masking. Say, for example, masking out central part of the parking signs and matching with what is remaining on the outside of the template. Depending on how your samples look, this should give your sufficient "hits". Of course you might have to work a little harder with problems like perspective correction and scaling. — Knight Forked

Twenkid Twenkid · Accepted Answer · 2020-12-20T04:04:56

What about starting with OCR, Tesseract, in order to recognize big "P" and other parking-related text patterns?

(Toast seems like Android: How can I use Tesseract in Android?

General Tesseract for Java: https://www.geeksforgeeks.org/tesseract-ocr-with-java-with-examples/ )

Another example, in Python, but see the preprocessing and other tricks and ideas for making the letters recognizable when the image has gradients, lower contrast, small fonts etc.: How to obtain the best result from pytesseract?

Also, there could be filtering by color, since the colors of the signs are known. The conversion to grayscale removes that valuable information, so finding the edges is OK, but the colors still can be used. E.g. split the colors to b,g,r and use each channel as grayscale and possibly boost it. The red and blue borders would stand out.

It seems the contrast around the red borders is too low, the blue signs are brighter compared to the black contour. If not splitting, before converting to grayscale, some of the color channels could be amplified anyway, like the red one.

Searching for big yellow/blue regions with low contrast, with text found, "P" etc. Tesseract has a function returning the boxes of the text that was found.

Also once you find a sign somewhere or a bar of signs and their directions, you could search there, vertically/horizontally.

You may search HoughLines as well, that may find the black border around the signs.

Calculate the perimeter with arcLength and approximate the contour with approxPolyDP

If approximated figure has 4 points, then assuming it is a rectangle hence adding the contour

IMO finding exactly 4 points (or after simplification of the polygon) is hard and may be not enough of an evidence, also there are round corners etc. if contours are compared directly.

The angles between the vertices and the distances matter - are the lines parallel (with some precision) etc.

The process could be iterative: gradually reducing the polygon detail, checking the area and perimeter, until the vertices reach 4 (or about that). If the area and perimeter don't change much (the ratio has to be found) after polygon aproximation (simplifying the round corners etc.), while the number of points in the contour gets reduced. I'd try also a comparison to the bounding box and the convex hull measurements etc.

How to detect parking signs using OpenCV?

2 Answers