0 Introduction
Today, the senior introduces a machine vision project to you
Image correction based on machine vision (taking license plate recognition as an example)
Bi design help, problem opening guidance, technical solutions ðŸ‡¶746876041
1 Introduction to ideas
At present, the license plate recognition system can be seen everywhere at the door of each community, and the recognition effect seems to be OK. After consulting the data, it is found that the whole process can be subdivided into four parts: license plate location, distortion correction, license plate segmentation and content recognition. This essay mainly introduces two parts: license plate location and distortion correction, which are realized by opencv in python environment.
1.1 license plate location
At present, the mainstream license plate location methods can be divided into two categories: one is based on the background color feature of license plate; Another method is based on the contour shape feature of license plate. Based on color features can be divided into two categories: one in RGB space and the other in HSV space. After testing, it is found that the effect of using any method alone is not ideal. At present, it is common to use several positioning methods at the same time, or use one recognition and the other verification. In this paper, the license plate is located mainly through color features, mainly based on the H component of HSV space, supplemented by the R component and B component of RGB space, and then the length width ratio of the license plate is used to eliminate interference.
1.2 distortion correction
In the process of license plate image acquisition, the camera lens is usually not perpendicular to the license plate, so the license plate in the image to be recognized will be distorted to a certain extent, which brings some difficulties to the subsequent license plate content recognition. Therefore, it is necessary to correct the distortion of the license plate to eliminate the adverse effects of distortion.
2 code implementation
2.1 license plate location
2.1.1 select suspicious areas through color features
The images of the license plate under different lighting environments are taken, the background color is intercepted, and the channel separation and color space conversion are carried out by opencv. After the test, the following characteristics of the background color of the license plate are summarized:

(1) In HSV space, the value of H component usually hovers around 115, and the s component and V component vary greatly due to different illumination (the value range of H component in opencv is 0 to 179, not 0 to 360 in iconography; the value range of S component and V component is 0 to 255);

(2) In RGB space, the R component is usually small, generally below 30, the B component is usually large, generally above 80, and the G component fluctuates greatly;

(3) In the HSV space, the image is filled with light and saturated, that is, the S component and V component of the image are set to 255, and then the color space is converted from HSV space to RGB space. It is found that all R components become 0 and all B components become 255 (this operation will introduce large interference and will not be used later).
According to the above features, the suspicious license plate area can be preliminarily screened. Then, the gray image is operated, and the pixel value of the suspicious position is set to 255 and the pixel value of other positions is set to 0, that is, the image is binarized according to the features. In the binarized image, the suspicious area is represented by white and other areas are black. Then, the image can be further processed through expansion and corrosion .
for i in range(img_h): for j in range(img_w): # Ordinary blue license plate, while excluding the interference of transparent reflective materials if ((img_HSV[:, :, 0][i, j]115)**2 < 15**2) and (img_B[i, j] > 70) and (img_R[i, j] < 40): img_gray[i, j] = 255 else: img_gray[i, j] = 0
2.1.2 find the peripheral contour of the license plate
After selecting the suspicious area and binarizing the image, generally, only the pixel color of the license plate position in the image is white, but there will be some noise in some special cases. As shown in the above figure, because there is a blue bracket in the upper right corner of the image, which is consistent with the color characteristics of the license plate, it is also recognized as a license plate, which introduces noise.
Through observation, it can be found that there are great differences between the license plate area and noise, and the characteristics of the license plate area are obvious:

(1) According to the shape of China's conventional license plate, the shape of the license plate is a flat rectangle, and the length width ratio is about 3:1;

(2) The area of the license plate is much larger than the noise area, which is generally the largest white area in the image.
You can find the outline of the white area in the binarized image through the cv2.findContours() function.
Note: in opencv2 and opencv4, there are two return values of cv2.findContours(), while in opencv3, there are three return values. Depending on the opencv version, the code writing method will also be different.
# Detect all outer contours, leaving only four vertices of the rectangle # opencv4.0, opencv2.x contours, _ = cv2.findContours(img_bin, cv2.RETR_LIST, cv2.CHAIN_APPROX_SIMPLE) # opencv3.x _, contours, _ = cv2.findContours(img_bin, cv2.RETR_LIST, cv2.CHAIN_APPROX_SIMPLE)
Here, because there are three white areas (license plate and two noises) in the binary image, the return value contours is a list with a length of 3. The list contains three arrays, and each array stores the contour information of a white area. The shape of each array is (n, 1, 2), that is, each array stores the coordinates of n points on the contour of the white area.
At present, we have obtained three array s, that is, three groups of contour information, but we do not know which is the group of contour information corresponding to the license plate area. At this time, we can filter the contour of the license plate area according to the above characteristics of the license plate.
#Shape and size screening verification det_x_max = 0 det_y_max = 0 num = 0 for i in range(len(contours)): x_min = np.min(contours[i][ :, :, 0]) x_max = np.max(contours[i][ :, :, 0]) y_min = np.min(contours[i][ :, :, 1]) y_max = np.max(contours[i][ :, :, 1]) det_x = x_max  x_min det_y = y_max  y_min if (det_x / det_y > 1.8) and (det_x > det_x_max ) and (det_y > det_y_max ): det_y_max = det_y det_x_max = det_x num = i # Get the contour point set of the most suspicious area points = np.array(contours[num][:, 0])
The shape of the final points is (n, 2), which stores the coordinates of n points, which are distributed on the edge of the license plate
2.1.3 location of license plate area
After obtaining the point set on the contour of the license plate, you can use CV2. Minaarearect() to obtain the minimum circumscribed rectangle of the point set. The return value rect contains the center point coordinates, height, width, inclination angle and other information of the rectangle. You can use cv2.boxPoints() to obtain the coordinates of the four vertices of the rectangle.
# Obtain the minimum circumscribed matrix, center point coordinates, width, height and rotation angle rect = cv2.minAreaRect(points) # Gets the four vertices of a rectangle, floating point box = cv2.boxPoints(rect) # Rounding box = np.int0(box)
However, we do not know which vertex of the rectangle these four coordinate points correspond to, so we can not make full use of these coordinate information.
Starting from the size characteristics of coordinate values, the four coordinates can be matched with the four vertices of the rectangle: in the opencv coordinate system, the smallest ordinate is top_point, the largest ordinate is bottom_point, the smallest abscissa is left_point, and the largest abscissa is right_point.
# Get four vertex coordinates left_point_x = np.min(box[:, 0]) right_point_x = np.max(box[:, 0]) top_point_y = np.min(box[:, 1]) bottom_point_y = np.max(box[:, 1]) left_point_y = box[:, 1][np.where(box[:, 0] == left_point_x)][0] right_point_y = box[:, 1][np.where(box[:, 0] == right_point_x)][0] top_point_x = box[:, 0][np.where(box[:, 1] == top_point_y)][0] bottom_point_x = box[:, 0][np.where(box[:, 1] == bottom_point_y)][0] # Up, down, left and right coordinates vertices = np.array([[top_point_x, top_point_y], [bottom_point_x, bottom_point_y], [left_point_x, left_point_y], [right_point_x, right_point_y]])
2.2 distortion correction
2.2.1 vertex location of license plate after distortion
In order to realize the distortion correction of license plate, the position relationship of corresponding points before and after distortion must be found.
It can be seen that the rectangular license plate becomes a parallelogram after distortion, so the license plate contour does not match the obtained rectangular contour. However, with the four vertex coordinates of the rectangle, the four vertex coordinates of the parallelogram license plate can be obtained through a simple geometric similarity relationship.
In this example, the four vertices of the parallelogram and the four vertices of the rectangle have the following relationship: rectangular vertex Top_Point,Bottom_Point and parallelogram vertex new_top_point,new_bottom_point coincident, rectangular vertex top_ Abscissa of point and parallelogram vertex NEW_ right_ The abscissa of point is the same, and the rectangular vertex bottom_ Abscissa of point and parallelogram vertex NEW_ left_ The abscissa of point is the same.
But in fact, due to different shooting angles, there may be two different distortions. The specific distortion can be judged according to the different inclination angles of the rectangle.
After judging the specific distortion, the coordinates of the four vertices of the parallelogram can be easily obtained by selecting the corresponding geometric similarity relationship, that is, the coordinates of the four vertices of the distorted license plate can be obtained.
In order to correct the license plate, the coordinates of the four vertices of the license plate before distortion need to be obtained. Because the standard size of the license plate in China is 440X140, the four vertex coordinates of the license plate before distortion can be specified as: (0,0), (440,0), (0140), (440140). The order should correspond to the four vertex coordinates after distortion.
# Distortion condition 1 if rect[2] > 45: new_right_point_x = vertices[0, 0] new_right_point_y = int(vertices[1, 1]  (vertices[0, 0] vertices[1, 0]) / (vertices[3, 0]  vertices[1, 0]) * (vertices[1, 1]  vertices[3, 1])) new_left_point_x = vertices[1, 0] new_left_point_y = int(vertices[0, 1] + (vertices[0, 0]  vertices[1, 0]) / (vertices[0, 0]  vertices[2, 0]) * (vertices[2, 1]  vertices[0, 1])) # Corrected four vertex coordinates point_set_1 = np.float32([[440, 0],[0, 0],[0, 140],[440, 140]]) # Distortion condition 2 elif rect[2] < 45: new_right_point_x = vertices[1, 0] new_right_point_y = int(vertices[0, 1] + (vertices[1, 0]  vertices[0, 0]) / (vertices[3, 0]  vertices[0, 0]) * (vertices[3, 1]  vertices[0, 1])) new_left_point_x = vertices[0, 0] new_left_point_y = int(vertices[1, 1]  (vertices[1, 0]  vertices[0, 0]) / (vertices[1, 0]  vertices[2, 0]) * (vertices[1, 1]  vertices[2, 1])) # Corrected four vertex coordinates point_set_1 = np.float32([[0, 0],[0, 140],[440, 140],[440, 0]]) # Coordinates of four vertices of parallelogram before correction new_box = np.array([(vertices[0, 0], vertices[0, 1]), (new_left_point_x, new_left_point_y), (vertices[1, 0], vertices[1, 1]), (new_right_point_x, new_right_point_y)]) point_set_0 = np.float32(new_box)
2.2.2 correction
This distortion is caused by the projection caused by the camera not perpendicular to the license plate, so it can be corrected by CV2. Warpprective().
# Transformation matrix mat = cv2.getPerspectiveTransform(point_set_0, point_set_1) # Projection transformation lic = cv2.warpPerspective(img, mat, (440, 140))
# Created by Dan Cheng senior: Q746876041 import cv2 import numpy as np # Pretreatment def imgProcess(path): img = cv2.imread(path) # Uniform size img = cv2.resize(img, (640,480)) # Gaussian blur img_Gas = cv2.GaussianBlur(img,(5,5),0) # RGB channel separation img_B = cv2.split(img_Gas)[0] img_G = cv2.split(img_Gas)[1] img_R = cv2.split(img_Gas)[2] # Read gray image and HSV space image img_gray = cv2.cvtColor(img_Gas, cv2.COLOR_BGR2GRAY) img_HSV = cv2.cvtColor(img_Gas, cv2.COLOR_BGR2HSV) return img, img_Gas, img_B, img_G, img_R, img_gray, img_HSV # Preliminary identification def preIdentification(img_gray, img_HSV, img_B, img_R): for i in range(480): for j in range(640): # Ordinary blue license plate, while excluding the interference of transparent reflective materials if ((img_HSV[:, :, 0][i, j]115)**2 < 15**2) and (img_B[i, j] > 70) and (img_R[i, j] < 40): img_gray[i, j] = 255 else: img_gray[i, j] = 0 # Define core kernel_small = np.ones((3, 3)) kernel_big = np.ones((7, 7)) img_gray = cv2.GaussianBlur(img_gray, (5, 5), 0) # Gaussian smoothing img_di = cv2.dilate(img_gray, kernel_small, iterations=5) # Corrosion for 5 times img_close = cv2.morphologyEx(img_di, cv2.MORPH_CLOSE, kernel_big) # Closed operation img_close = cv2.GaussianBlur(img_close, (5, 5), 0) # Gaussian smoothing _, img_bin = cv2.threshold(img_close, 100, 255, cv2.THRESH_BINARY) # Binarization return img_bin # location def fixPosition(img, img_bin): # Detect all outer contours, leaving only four vertices of the rectangle contours, _ = cv2.findContours(img_bin, cv2.RETR_LIST, cv2.CHAIN_APPROX_SIMPLE) #Shape and size screening verification det_x_max = 0 det_y_max = 0 num = 0 for i in range(len(contours)): x_min = np.min(contours[i][ :, :, 0]) x_max = np.max(contours[i][ :, :, 0]) y_min = np.min(contours[i][ :, :, 1]) y_max = np.max(contours[i][ :, :, 1]) det_x = x_max  x_min det_y = y_max  y_min if (det_x / det_y > 1.8) and (det_x > det_x_max ) and (det_y > det_y_max ): det_y_max = det_y det_x_max = det_x num = i # Get the contour point set of the most suspicious area points = np.array(contours[num][:, 0]) return points #img_lic_canny = cv2.Canny(img_lic_bin, 100, 200) def findVertices(points): # Obtain the minimum circumscribed matrix, center point coordinates, width, height and rotation angle rect = cv2.minAreaRect(points) # Gets the four vertices of a rectangle, floating point box = cv2.boxPoints(rect) # Rounding box = np.int0(box) # Get four vertex coordinates left_point_x = np.min(box[:, 0]) right_point_x = np.max(box[:, 0]) top_point_y = np.min(box[:, 1]) bottom_point_y = np.max(box[:, 1]) left_point_y = box[:, 1][np.where(box[:, 0] == left_point_x)][0] right_point_y = box[:, 1][np.where(box[:, 0] == right_point_x)][0] top_point_x = box[:, 0][np.where(box[:, 1] == top_point_y)][0] bottom_point_x = box[:, 0][np.where(box[:, 1] == bottom_point_y)][0] # Up, down, left and right coordinates vertices = np.array([[top_point_x, top_point_y], [bottom_point_x, bottom_point_y], [left_point_x, left_point_y], [right_point_x, right_point_y]]) return vertices, rect def tiltCorrection(vertices, rect): # Distortion condition 1 if rect[2] > 45: new_right_point_x = vertices[0, 0] new_right_point_y = int(vertices[1, 1]  (vertices[0, 0] vertices[1, 0]) / (vertices[3, 0]  vertices[1, 0]) * (vertices[1, 1]  vertices[3, 1])) new_left_point_x = vertices[1, 0] new_left_point_y = int(vertices[0, 1] + (vertices[0, 0]  vertices[1, 0]) / (vertices[0, 0]  vertices[2, 0]) * (vertices[2, 1]  vertices[0, 1])) # Corrected four vertex coordinates point_set_1 = np.float32([[440, 0],[0, 0],[0, 140],[440, 140]]) # Distortion condition 2 elif rect[2] < 45: new_right_point_x = vertices[1, 0] new_right_point_y = int(vertices[0, 1] + (vertices[1, 0]  vertices[0, 0]) / (vertices[3, 0]  vertices[0, 0]) * (vertices[3, 1]  vertices[0, 1])) new_left_point_x = vertices[0, 0] new_left_point_y = int(vertices[1, 1]  (vertices[1, 0]  vertices[0, 0]) / (vertices[1, 0]  vertices[2, 0]) * (vertices[1, 1]  vertices[2, 1])) # Corrected four vertex coordinates point_set_1 = np.float32([[0, 0],[0, 140],[440, 140],[440, 0]]) # Coordinates of four vertices of parallelogram before correction new_box = np.array([(vertices[0, 0], vertices[0, 1]), (new_left_point_x, new_left_point_y), (vertices[1, 0], vertices[1, 1]), (new_right_point_x, new_right_point_y)]) point_set_0 = np.float32(new_box) return point_set_0, point_set_1, new_box def transform(img, point_set_0, point_set_1): # Transformation matrix mat = cv2.getPerspectiveTransform(point_set_0, point_set_1) # Projection transformation lic = cv2.warpPerspective(img, mat, (440, 140)) return lic def main(): path = 'F:\\Python\\license_plate\\test\\9.jpg' # Image preprocessing img, img_Gas, img_B, img_G, img_R, img_gray, img_HSV = imgProcess(path) # Preliminary identification img_bin = preIdentification(img_gray, img_HSV, img_B, img_R) points = fixPosition(img, img_bin) vertices, rect = findVertices(points) point_set_0, point_set_1, new_box = tiltCorrection(vertices, rect) img_draw = cv2.drawContours(img.copy(), [new_box], 1, (0,0,255), 3) lic = transform(img, point_set_0, point_set_1) # The license plate is framed on the original drawing cv2.namedWindow("Image") cv2.imshow("Image", img_draw) # Binary image cv2.namedWindow("Image_Bin") cv2.imshow("Image_Bin", img_bin) # Display corrected license plate cv2.namedWindow("Lic") cv2.imshow("Lic", lic) # Pause, close window cv2.waitKey(0) cv2.destroyAllWindows() if __name__ == '__main__': main()
7 finally  design help
Bi design help, problem opening guidance, technical solutions ðŸ‡¶746876041