TL;DR
This paper introduces PP-OCRv2, an improved ultra-lightweight OCR system that significantly boosts accuracy without sacrificing efficiency. By incorporating a "bag of tricks" including novel data augmentation and advanced training strategies for both text detection and recognition, PP-OCRv2 achieves a 7% precision improvement over its predecessor while remaining comparable to larger, server-grade OCR models.
What Problem Does This Paper Address?
Optical Character Recognition (OCR) systems are crucial for digitizing documents and extracting information from images. However, building effective OCR systems involves a trade-off between accuracy and computational efficiency, especially for deployment on resource-constrained devices or in real-time applications. The authors previously developed PP-OCR, a practical ultra-lightweight OCR system that balanced these factors. This paper addresses the challenge of further improving the accuracy of such lightweight systems while maintaining their efficiency, aiming to close the gap with more powerful, but heavier, server-based OCR solutions.
Methods
PP-OCRv2 builds upon the PP-OCR framework by introducing several new "tricks" for training both the text detection and text recognition components.
For Text Detection:
- Collaborative Mutual Learning (CML): This technique addresses limitations in knowledge distillation for text detection. It uses two "student" networks that learn from each other (via a Deep Mutual Learning approach) and are guided by a "teacher" network. This allows the student models to potentially surpass the teacher's accuracy.
- CopyPaste: A data augmentation strategy adapted from object detection and instance segmentation. It involves synthesizing new text instances by pasting them onto existing backgrounds, which helps balance training data and improve robustness.
For Text Recognition:
- Lightweight CPU Network (PP-LCNet): A new lightweight backbone designed for Intel CPUs, optimized for speed using MKLDNN. It's based on MobileNetV1 but incorporates modifications like improved activation functions, strategically placed SE modules, larger convolution kernels, and an enhanced 1x1 convolutional layer after Global Average Pooling (GAP) to boost feature extraction capabilities.
- Unified-Deep Mutual Learning (U-DML): Similar to CML, this method uses two identical student networks that learn from each other without requiring a pre-trained teacher. It supervises not only the final output but also the intermediate feature maps, promoting consistency.
- Enhanced CTCLoss: This modification to the Connectionist Temporal Classification (CTC) loss incorporates the idea of CenterLoss to better distinguish between visually similar characters, which is particularly useful for languages like Chinese.
Datasets
The experiments were conducted on datasets similar to those used in the previous PP-OCR work.
- Text Detection: 97k training images (68k real, 29k synthetic) and 500 validation images (all real). The real scene images are sourced from public datasets like LSVT, RCTW-17, MTWI 2018, CASIA-10K, SROIE, MLT 2019, BDI, MSRA-TD500, and CCPD 2019.
- Text Recognition: 17.9M training images (1.9M real, 16M synthetic) and 18.7k validation images (all real). Real scene images come from public datasets and web searches, while synthetic images cover various challenging scenarios.
- Evaluation: An additional 300 images from diverse real-world applications were collected for overall system evaluation.
Key Insights and Ideas
- "Bag of Tricks" for Lightweight OCR: The paper demonstrates that combining multiple, seemingly disparate techniques can lead to substantial improvements in lightweight OCR systems.
- Mutual Learning for Robustness: CML and U-DML highlight the power of self-distillation and mutual learning between networks to improve robustness and accuracy, even surpassing teacher models in some cases.
- Data Augmentation's Broad Applicability: CopyPaste, originally successful in object detection, proves effective for text detection, showcasing the transferability of augmentation techniques across vision tasks.
- CPU-Specific Optimization: The development of PP-LCNet underscores the importance of hardware-aware network design for achieving optimal performance on specific platforms like Intel CPUs.
- Addressing Character Ambiguity: The Enhanced CTCLoss directly tackles a common OCR challenge: distinguishing visually similar characters, leading to better recognition accuracy.
Key Results
- Accuracy Improvement: PP-OCRv2 achieves a 7% higher precision compared to the original PP-OCR under the same inference cost.
- Server-Level Performance: The accuracy of PP-OCRv2 is comparable to the larger, server-based PP-OCR models that use ResNet backbones.
- Text Detection Gains: Ablation studies show DML improves Hmean by ~2%, CML by ~3%, and CopyPaste by an additional 0.6%, resulting in a total 3.6% improvement over the PP-OCR detection baseline.
- Text Recognition Gains: PP-LCNet offers a 2.6% accuracy improvement over MobileNetV3 with a faster inference time (6.2ms vs. 7.7ms). U-DML further boosts accuracy by 4.6%, and Enhanced CTCLoss adds another 0.9%, for a total of 8.1% improvement.
- Inference Speed: Despite accuracy gains, PP-OCRv2 maintains competitive inference speeds. PP-LCNet is 1.5ms faster than MobileNetV3, and the overall PP-OCRv2 mobile system achieves 11.6ms inference time on CPU, comparable to the original PP-OCR mobile.
Limitations and Future Work
- PP-OCRv2-tiny: The authors mention that strategies in the gray boxes of Figure 2 (which were effective in PP-OCR but not explicitly validated in this paper) will be adopted in the future to further speed up the inference of PP-OCRv2-tiny.
- Model Size: While PP-LCNet offers speed benefits, its model size is 3M larger than MobileNetV3. The paper notes that the overall PP-OCRv2 mobile system has a slightly larger model size (0.576M) compared to the original PP-OCR mobile (0.503M), but this is justified by the significant accuracy and speed improvements.
- Future Work: The paper implicitly suggests continued exploration of these "bag of tricks" and their application to even more lightweight models (PP-OCRv2-tiny). The open-sourcing of the models and code also invites further community research and development.
Summary updated