This blog will introduce how to use template matching to find objects in images. Template matching is a method to find the position of template image in a large image. OpenCV uses cv2.matchTemplate() and cv2.minMaxLoc() to realize template matching.

It is mainly divided into three parts:

1. Template matching single object
2. Template matching multiple objects
3. Multiscale template matching (this paper uses a template with the same size and size as the object to be detected in the original image; it can be optimized here, that is, the enlargement and reduction of the template image can be detected. For reference: Multiscale template matching using Python and OpenCV)

1. Renderings

1.1 template matching single object

cv2.TM_CCOEFF renderings are as follows:

CV2. Tm_ccoeff_normalized effect drawing is as follows:
cv2.TM_CCORR:
It can be seen that the effect of cv2.TM_CCORR is not good, and the template is not successfully detected;
cv2.TM_CCORR_NORMED:
cv2.TM_SQDIFF:
cv2.TM_SQDIFF_NORMED:

1.2 template matching multiple objects

The coin template is shown as follows:

The original image VS multiple objects (coin template) have been successfully detected. The effect diagram is as follows:
It can be seen that there are 6 Coin templates in the original picture of super Mary, which are successfully detected (red boxes are drawn on the coin object);

1.3 multi scale template matching

When the template size is inconsistent with that in the original drawing, it can also be matched. The effect diagram is as follows:

2. Principle

OpenCV uses cv2.matchTemplate() and cv2.minMaxLoc() to realize template matching. It slides the template image onto the input image (such as two-dimensional convolution), and compares the input image under the template image;

res = cv2.matchTemplate(img, template, method)

• img: original grayscale image
• Template: template grayscale image
• Methods: 6, including cv2.TM_CCOEFF, cv2.TM_CCOEFF_normalized, cv2.TM_CCORR, cv2.TM_CCORR_normalized, cv2.TM_SQDIFF, cv2.TM_SQDIFF_normalized;
• res: it returns a grayscale image, where each pixel represents the degree to which the neighborhood of the pixel matches the template.

If the size of the input image is (WXH) and the size of the template image is (WXH), the size of the output image is (W-w+1, H-h+1). You can use the cv2.minMaxLoc() function to find the position of the maximum / minimum value. Take it as the upper left corner of the rectangle and (w, h) as the width and height of the rectangle. This rectangle is the template area.

In this paper, a template with the same size and size as the original graph object is used for matching;
It can be optimized here, that is, the enlargement and reduction of the template image can be detected. Refer to: Multiscale template matching using Python and OpenCV

3. Source code

3.1 template matching single object

# Using Python and opencv for template matching to find objects
# Template matching single object

import cv2
from matplotlib import pyplot as plt

img = cv2.imread('zly.jpg', 0)
img2 = img.copy()
template = cv2.imread('zly_template.jpg', 0)
w, h = template.shape[::-1]

# Matching method in 6
methods = ['cv2.TM_CCOEFF', 'cv2.TM_CCOEFF_NORMED', 'cv2.TM_CCORR',
           'cv2.TM_CCORR_NORMED', 'cv2.TM_SQDIFF', 'cv2.TM_SQDIFF_NORMED']

for meth in methods:
    img = img2.copy()
    method = eval(meth)

    # Apply template matching
    # If the size of the input image is (WXH) and the size of the template image is (WXH), the size of the output image is (W-w+1, H-h+1).
    # Once you get the result, you can use the cv2.minMaxLoc() function to find the location of the maximum / minimum value.
    # Take it as the upper left corner of the rectangle and (w, h) as the width and height of the rectangle to get the template area.
    res = cv2.matchTemplate(img, template, method)
    min_val, max_val, min_loc, max_loc = cv2.minMaxLoc(res)

    # If the method is TM_SQDIFF or TM_SQDIFF_normalized, take the minimum value
    if method in [cv2.TM_SQDIFF, cv2.TM_SQDIFF_NORMED]:
        top_left = min_loc
    else:
        top_left = max_loc
    bottom_right = (top_left[0] + w, top_left[1] + h)

    cv2.rectangle(img, top_left, bottom_right, 255, 2)

    plt.subplot(121), plt.imshow(res, cmap='gray')
    plt.title('Matching Result'), plt.xticks([]), plt.yticks([])
    plt.subplot(122), plt.imshow(img, cmap='gray')
    plt.title('Detected Point'), plt.xticks([]), plt.yticks([])
    plt.suptitle(meth)

    plt.show()

3.2 template matching multiple objects

# Template matching multiple objects (detect multiple coin template objects in super Mary)

import cv2
import numpy as np

img_rgb = cv2.imread('cjml.jpg')
cv2.imshow("origin",img_rgb)
img_gray = cv2.cvtColor(img_rgb, cv2.COLOR_BGR2GRAY)
template = cv2.imread('cjml_template.jpg')
cv2.imshow("template",template)
template = cv2.cvtColor(template, cv2.COLOR_BGR2GRAY)

w, h = template.shape[::-1]

res = cv2.matchTemplate(img_gray, template, cv2.TM_CCOEFF_NORMED)
threshold = 0.8
loc = np.where(res >= threshold)
for pt in zip(*loc[::-1]):
    cv2.rectangle(img_rgb, pt, (pt[0] + w, pt[1] + h), (0, 0, 255), 2)

cv2.imwrite('cjml_res.jpg', img_rgb)
cv2.imshow("cjml_res",img_rgb)
cv2.waitKey(0)

reference resources

• https://docs.opencv.org/3.0-beta/doc/py_tutorials/py_imgproc/py_template_matching/py_template_matching.html#py-template-matching