SamLP: A Customized Segment Anything Model for License Plate Detection

Haoxuan Ding, Junyu Gao, Yuan Yuan, Qi WangarXiv 2024 · DOI · arXiv

Introduction

License plate (LP) detection is a critical component of intelligent transportation systems. While existing solutions are mature, they often struggle with "localization"—the inability to generalize across different countries due to variations in plate colors, fonts, and vehicle types. SamLP addresses this by leveraging the Segment Anything Model (SAM), a vision foundation model, to achieve state-of-the-art performance with minimal training data.

The Localization Problem

Traditional detectors like YOLO or FCOS require massive, diverse datasets to perform well across regions. A model trained on Brazilian plates often fails when deployed in China or Taiwan. SamLP utilizes the "world knowledge" inherent in SAM to bridge this gap, maintaining high accuracy even when data is scarce.

Example of a Brazilian license plate from the UFPR-ALPR dataset, illustrating the regional styles the model must learn.

Methodology

SamLP adapts the heavy SAM architecture through two primary innovations:

  1. LoRA Fine-tuning: To manage computational costs, the researchers employed Low-Rank Adaptation (LoRA). By freezing the original SAM parameters and injecting trainable rank decomposition matrices into the Transformer blocks, the model learns LP-specific features efficiently.
  2. Iterative Prompting: To preserve SAM's promptable nature, an iterative strategy was introduced. The model's initial segmentation mask serves as a "prior prompt" for a second pass, allowing the system to self-correct and refine boundaries in complex scenes.

Experimental Evaluation

The model was rigorously tested across four diverse benchmarks: UFPR-ALPR (Brazil), CCPD and CRPD (Mainland China), and AOLP (Taiwan). These datasets encompass a wide variety of environmental conditions and plate styles.

Diverse license plate styles from the UFPR-ALPR, CCPD, CRPD, and AOLP datasets used to evaluate SamLP's generalization.

Key Findings

  • Few-Shot Proficiency: SamLP demonstrates remarkable efficiency, achieving over 91% Average Precision (AP) with only 60 training images—a scenario where traditional models typically fail.
  • Zero-Shot Transfer: When trained on Brazilian data and tested on Chinese plates, SamLP achieved 90.3 AP, significantly outperforming FCOS (41.3 AP). This confirms that foundation models retain fundamental object recognition capabilities that transcend regional styles.
  • Architectural Insights: Ablation studies revealed that fine-tuning the image encoder is more critical than the mask decoder for distinguishing license plates from background noise.

Conclusion and Future Work

SamLP sets a new benchmark for robust, data-efficient license plate detection. While it shows slight performance drops in extreme low-light conditions compared to specialized detectors, its generalization capabilities are unparalleled. Future research will focus on knowledge distillation to compress SamLP into smaller, real-time models suitable for edge deployment.

Summary updated

BibTeX
@misc{ding2024samlp,
  author = {Haoxuan Ding and Junyu Gao and Yuan Yuan and Qi Wang},
  title = {{SamLP: A Customized Segment Anything Model for License Plate Detection}},
  year = {2024},
  eprint = {2401.06374},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2401.06374},
}
Figure