Abstract
Automatic License Plate Recognition (ALPR) has been a frequent topic of research due to many practical applications. However, many of the current solutions are still not robust in real-world situations, commonly depending on many constraints. This paper presents a robust and efficient ALPR system based on the state-of-the-art YOLO object detector. The Convolutional Neural Networks (CNNs) are trained and fine-tuned for each ALPR stage so that they are robust under different conditions (e.g., variations in camera, lighting, and background). Specially for character segmentation and recognition, we design a two-stage approach employing simple data augmentation tricks such as inverted License Plates (LPs) and flipped characters. The resulting ALPR approach achieved impressive results in two datasets. First, in the SSIG dataset, our system achieved a recognition rate of 93.53% and 47 Frames Per Second (FPS), performing better than both Sighthound and OpenALPR commercial systems. Second, targeting a more realistic scenario, we introduce a larger public dataset, called UFPR-ALPR dataset, where our system achieved a recognition rate of 78.33% and 35 FPS, significantly outperforming commercial alternatives.
Introduction
While Automatic License Plate Recognition (ALPR) is a mature field, most existing systems are "brittle," relying on strict constraints such as static cameras, specific viewing angles, or ideal lighting. When these systems encounter "in-the-wild" conditions—such as moving cameras, varied vehicle types (like motorcycles and buses), or tilted plates—their accuracy often plummets. Furthermore, many Deep Learning approaches suffer from a lack of diverse training data. The authors set out to build a system robust enough for real-time use on mobile cameras while handling the complexities of urban traffic.
Methodology
The proposed system utilizes a multi-stage pipeline where each step is powered by a version of the YOLO (You Only Look Once) detector, specifically tuned for the task.
Vehicle and License Plate Detection
The system first identifies the vehicle and then locates the license plate (LP) within that vehicle's bounding box. The authors tested both Fast-YOLO for speed and YOLOv2 for higher accuracy. This hierarchical approach ensures that the LP search space is constrained to relevant regions of the image.
Character Segmentation and Recognition
Instead of traditional image processing, a modified YOLO-based architecture called CR-NET is used to find and isolate individual characters within the LP patch. To improve recognition accuracy, the process is split into two specialized CNNs: one for letters and one for digits. This leverages the known format of Brazilian LPs (3 letters followed by 4 digits) to reduce confusion between similar-looking characters like '0' and 'O'.
Data Augmentation and Temporal Redundancy
To handle different plate colors (e.g., gray for private cars, red for buses), the authors used "inverted" LPs and flipped characters during training. Furthermore, since the system processes video, it tracks a vehicle across multiple frames and uses a majority voting scheme to decide on the final license plate string, filtering out transient errors.
Datasets and Experimental Results
The study utilizes two primary benchmarks:
- SSIG SegPlate Database: A standard dataset of 2,000 frames featuring a static camera and consistent backgrounds.
- UFPR-ALPR Dataset: A new, more challenging dataset introduced by the authors. It contains 4,500 frames from 150 videos captured by moving cameras, including diverse vehicle types (cars, motorcycles, buses, trucks) and complex backgrounds.
On the SSIG dataset, the system achieved a 93.53% recognition rate at 47 FPS, outperforming commercial systems like Sighthound (89.80%) and OpenALPR (93.03%). In the more difficult UFPR-ALPR "moving camera" scenario, commercial systems struggled (below 70%), while the proposed system maintained a 78.33% recognition rate at 35 FPS.
Conclusion
The use of YOLO across all stages of the ALPR pipeline proves highly effective for real-time applications. While the system outperforms leading commercial solutions, the UFPR-ALPR dataset highlights remaining challenges in unconstrained environments, such as highly tilted plates and low-light conditions. Future work will focus on plate rectification and layout-independent recognition to improve international applicability.
Summary updated