Presented at AAAI-2024 by researchers from Shopee Pte. Ltd., the Complementary Proposal Network (CPN) introduces a novel architecture designed to resolve the long-standing "either-or" dilemma in scene text detection. By fusing the strengths of segmentation-based and anchor-based approaches, CPN achieves state-of-the-art results on curved and multi-oriented text benchmarks while remaining faster and lighter than standard models like Mask R-CNN.
The Core Dilemma in Scene Text Detection
Detecting text in the wild is notoriously difficult because text instances are rarely simple rectangles; they can be curved, extremely long, or tightly packed. Historically, researchers have relied on two distinct paradigms:
- Segmentation-based: These methods identify text pixels, making them excellent for arbitrary shapes like curved text. However, they often struggle when text lines are so close that they "bleed" into each other during post-processing.
- Anchor-based: These use predefined boxes (anchors) to localize text. They are superior at separating crowded text but often fail when text doesn't fit standard boxy shapes, such as long, winding banners.
The authors posit that these two methods are inherently complementary. The CPN architecture is designed to integrate them efficiently without creating a slow, bloated model.
Methodology: The CPN Architecture
The CPN architecture consists of three primary innovations that allow for parallel processing of semantic shapes and geometric structures:
- Deformable Morphology Semantic Network (Deformable MSN): This branch addresses the "shape" problem. Instead of traditional, slow CPU-based post-processing to group pixels, it utilizes a Deformable Dilation Layer. This allows the network to learn how to expand a predicted text core into a full text instance entirely on the GPU, enhancing both flexibility and speed.
- Balanced Region Proposal Network (Balanced RPN): This branch handles the layout using traditional anchors. Because the MSN branch manages complex shapes, the authors "balance" this branch by reducing the number of proposals to 300, which minimizes false positives and accelerates the model.
- Interleaved Feature Attention (IFA): This module acts as the "glue" connecting the two branches. It enables semantic features (appearance) and geometric features (location) to interact before proposals are finalized. Crucially, IFA reduces feature channels from 256 to 128, lowering the total computational cost (FLOPs) compared to a standard backbone.
Key Insights and Efficiency
A critical finding of the research is the relationship between Intersection-over-Union (IoU) thresholds and detection accuracy. The Semantic branch (MSN) excels at high IoU thresholds, providing precise shape boundaries. Conversely, the Geometric branch (RPN) performs better at lower IoU thresholds, identifying general areas in complex layouts. By combining them, CPN maintains high recall across all thresholds.
Furthermore, by converting morphological operations like dilation into differentiable neural network layers, the team removed the CPU bottlenecks common in segmentation-based detectors. Despite the dual-branch approach, CPN is more efficient than a single-branch Mask R-CNN due to the channel reduction in the IFA module and the thinned-out RPN.
Experimental Results
CPN was evaluated on five major benchmarks, setting new state-of-the-art (SOTA) records:
- IC19-ArT: Achieved an F-measure of 81.7%, a 3.6% improvement over previous methods.
- IC15: Reached 90.4% F-measure, outperforming recent models like FSG.
- MSRA-TD500: Reached 89.9% F-measure.
- Computational Performance: CPN runs at 11.2 FPS on IC15, faster than Mask R-CNN (9.6 FPS), while using fewer parameters (37.38M vs 43.75M) and fewer GFLOPs (99.61 vs 142.39).
Conclusion and Future Work
CPN successfully bridges the gap between semantic and geometric text detection. While the current model focuses on detection, future work may involve integrating it into end-to-end "Spotter" systems (Detection + Recognition) or applying this complementary approach to general object detection tasks beyond text.
Summary updated