Click on "Xiaobai Learns Vision" above, select to add "Star" or "Top"
Heavyweight content delivered first

License Plate Recognition

Overview

License plate recognition based on deep learning, where the vehicle detection network directly uses YOLO for detection. Then, a network is used to detect and recognize the license plate number.

The license plate detection network uses ResNet18, and the network outputs the affine transformation matrix of the detected bounding box, capable of detecting quadrilaterals of arbitrary shapes.

The license plate number sequence model adopts the ResNet18 + Transformer model, which directly outputs the license plate number sequence.

For the dataset, the license plate detection uses the CCPD 2019 dataset. During the training of the detection model, the program generates fake license plates and overlays them on the dataset images to enhance detection capability.

The sequence recognition of the license plate number directly uses the program-generated license plate images for training, supplemented with appropriate image enhancement techniques. The model training adopts an end-to-end training approach, where input images directly output the license plate number sequence, with loss calculated using CTCLoss.

1. Network Model

1. License Plate Detection Network Model:

The network code is defined as follows:

This network essentially divides the image into cells, detecting license plates within a 16X16 grid and outputting the reflection transformation matrix of the detected license plate.

2. License Plate Number Sequence Recognition Network:

The backbone network for license plate number sequence recognition uses ResNet18 + Transformer, where ResNet18 encodes the image, and the transformer decodes it into corresponding characters.

The network code is defined as follows:

The code for the Block class is as follows:

The code for positional encoding is as follows:

The self-attention code used in the Block class is as follows:

2. Data Loading

1. Loading License Plate Numbers

Generate a set of license plate numbers through the program:

Then perform data augmentation,

Mainly including:

3. Training

Train them separately

Among them, the loss calculation for the detection network is as follows:

The detection network outputs the reflection transformation matrix, but the labels for the license plate position are given as the positions of the four corners, so loss needs to be calculated after the response conversion. Among them, whether the cell has a target is calculated using CrossEntropyLoss, while the loss for the license plate position uses L1Loss.

4. Inference

1. Inference of the Detection Network

Inference can be performed as with a typical detection network. However, an additional step is required to convert the reflection transformation matrix into bounding box position calculations.

Additionally, when the YOLO detects the measurement image for license plate detection, an operation is performed. The code is shown below, extracting the vehicle detection box image and resizing it to an integer multiple of 16 for both width and height.

2. Inference of the Sequence Detection Network

For the output sequence of the network, perform deduplication. For example, when the interval identifier is “*”:

Complete code

https://github.com/HibikiJie/LicensePlate

Original article address

https://blog.csdn.net/weixin_48866452/article/details/120319588

Download 1: OpenCV-Contrib Extension Module Chinese Version Tutorial

Reply "Extension Module Chinese Tutorial" in the "Xiaobai Learns Vision" public account backend to download the first OpenCV extension module tutorial in Chinese, covering installation of extension modules, SFM algorithms, stereo vision, object tracking, biological vision, super-resolution processing, and more than twenty chapters of content.

Download 2: Python Vision Practical Projects 52 Lectures

Reply "Python Vision Practical Projects" in the "Xiaobai Learns Vision" public account backend to download 31 practical vision projects, including image segmentation, mask detection, lane line detection, vehicle counting, adding eyeliner, license plate recognition, character recognition, emotion detection, text content extraction, facial recognition, and more, to help quickly learn computer vision.

Download 3: OpenCV Practical Projects 20 Lectures

Reply "OpenCV Practical Projects 20 Lectures" in the "Xiaobai Learns Vision" public account backend to download 20 practical projects based on OpenCV, achieving advanced learning of OpenCV.

Group Chat

Welcome to join the public account reader group to communicate with peers. Currently, there are WeChat groups for SLAM, 3D vision, sensors, autonomous driving, computational photography, detection, segmentation, recognition, medical imaging, GAN, algorithm competitions, etc. (which will gradually be subdivided in the future). Please scan the WeChat number below to join the group, and note: "Nickname + School/Company + Research Direction", for example: "Zhang San + Shanghai Jiaotong University + Vision SLAM". Please follow the format for the note, otherwise, it will not be accepted. After successfully adding, invitations will be sent to relevant WeChat groups based on research direction. Please do not send advertisements in the group, otherwise, you will be removed. Thank you for your understanding~