TL;DR
LeafText is a bio-inspired scene text detection framework that represents irregular text contours as "leaf margins" supported by a hierarchical structure of main, lateral, and thin veins. By simulating the growth process of a leaf, the model achieves state-of-the-art accuracy in fitting highly curved, ribbon-like text lines that traditional bounding-box or simple regression methods often miss.
What Problem Does This Paper Address?
Scene Text Detection (STD) is the task of identifying and localizing text in natural images—a critical step for autonomous systems and translation apps. While deep learning has mastered "regular" (rectangular) text, "irregular" text—curved, slanted, or wrapped around objects—remains a major hurdle.
Existing approaches generally fall into two camps: segmentation-based (which requires massive amounts of pixel-level data) and regression-based (which predicts point sequences). Regression methods often struggle with "ribbon-like" text because multiple contour points can end up in the same direction, leading to messy or overlapping results. Furthermore, many models fail to balance "multi-scale" text, often ignoring small text instances in favor of larger, easier-to-detect ones.
Methods
The authors introduce LeafText, a one-stage framework built on a novel Leaf Vein-based Text representation (LVT). The core idea is to treat the text contour as a leaf margin and reconstruct it through a three-tiered "vein" hierarchy:
- Main Vein (MV): Extracted from a predicted kernel mask, this acts as the "spine" of the text instance, providing a rough location and orientation.
- Lateral Veins (LV): These sprout perpendicularly from the main vein to determine the coarse boundaries of the text.
- Thin Veins (TV): These are shorter refinements that sprout from the lateral veins to "fine-tune" the final contour, ensuring high precision even for jagged or highly curved edges.
To make this work, the authors developed two key components:
- Multi-Oriented Smoother (MOS): Standard sampling often results in "jagged" center lines. MOS uses a rotating coordinate system and polynomial fitting to ensure the "main vein" is smooth and robust, providing a reliable foundation for the lateral veins.
- Global Incentive Loss ($L_g$): Instead of standard L2 loss, they use a negative logarithm loss that normalizes errors across different scales. This "incentivizes" the model to pay equal attention to tiny text and massive billboards.
Datasets
The model was pre-trained on SynthText (800k synthetic images) and evaluated on four major benchmarks:
- MSRA-TD500: Focuses on multi-oriented, line-level Chinese and English text.
- CTW1500: Contains 1,500 samples of line-level curved text.
- Total-Text: A dataset specifically for word-level arbitrary-shaped and curved text.
- ICDAR2015: A standard "incidental" scene text dataset with complex backgrounds.
Key Insights and Ideas
- Bionics as Geometry: The "aha moment" here is realizing that a leaf's vein structure is essentially a directed graph that naturally covers irregular geometries. By mimicking this, the authors solve the "directionality" problem that plagues other regression models.
- Polar vs. Cartesian: The model grows the main vein in Cartesian coordinates (standard x, y) but generates lateral and thin veins in polar coordinates. This allows the model to "emit" rays outward to find the text boundary more efficiently.
- The Power of "Thin Veins": The authors found that adding the "thin vein" layer—essentially a second level of refinement—reduced model complexity while accelerating training convergence. It’s a "divide and conquer" strategy for precision.
Key Results
LeafText outperformed existing state-of-the-art (SOTA) methods across the board:
- MSRA-TD500: Achieved an 87.8% F-measure, surpassing the previous best (ReLaText) by 1.1%.
- Total-Text: Reached 87.3% F-measure, a significant jump over traditional segmentation-based models.
- CTW1500: Hit 85.5% F-measure, proving its effectiveness on long, curved text lines.
- Efficiency: Because it is a one-stage framework, it avoids the computationally expensive post-processing (like complex grouping or NMS) required by many multi-stage detectors.
Limitations and Future Work
Despite its success, the authors note two primary challenges:
- False Positives: The model occasionally identifies non-text regions with high visual similarity to text (e.g., certain architectural patterns) as text instances.
- Text Adhesion: When two text lines are extremely close to each other, the "veins" can sometimes overlap or "over-emit," causing the model to struggle to distinguish between separate instances.
Future work will focus on improving the model's ability to distinguish between these interference regions and refining the "growth" process to prevent instance overlap.
Summary updated