Text Growing on Leaf

Chuang Yang, Mulin Chen, Yuan Yuan, Qi Wang

Abstract

Scene Text Detection (STD) for irregular, curved text remains a significant challenge for traditional bounding-box or simple regression methods. We present LeafText, a novel one-stage framework that models text contours as "leaf margins" supported by a hierarchical vein structure. By simulating leaf growth—from a main vein to lateral and thin veins—LeafText achieves high precision in fitting complex, ribbon-like text geometries. Experimental results on benchmarks like MSRA-TD500, CTW1500, and Total-Text demonstrate state-of-the-art performance, particularly in handling multi-scale and highly curved text.

Introduction

Identifying and localizing text in natural images is a foundational task for autonomous systems and translation applications. While deep learning has matured for regular rectangular text, "irregular" text—curved, slanted, or wrapped around objects—remains problematic. Existing regression-based methods often struggle with "ribbon-like" text where multiple contour points collapse into similar directions, leading to overlapping or inaccurate boundaries. Furthermore, balancing detection across various scales remains a hurdle, as many models favor larger, easier-to-detect instances.

Methods: Leaf Vein-based Text Representation (LVT)

LeafText introduces a bio-inspired representation that treats text contours as leaf margins. The reconstruction follows a three-tiered hierarchy that mimics natural growth:

  1. Main Vein (MV): Acts as the "spine" of the text instance, extracted from a predicted kernel mask to provide location and orientation.
  2. Lateral Veins (LV): These sprout perpendicularly from the main vein to establish coarse boundaries.
  3. Thin Veins (TV): Shorter refinements sprouting from lateral veins to fine-tune the final contour for high precision.
The hierarchical growth process of the Leaf Vein-based Text representation, from main vein to final leaf margin.

Technical Components

To ensure robustness, the framework employs a Multi-Oriented Smoother (MOS), which uses polynomial fitting and a rotating coordinate system to generate a smooth center line (Main Vein). This provides a reliable foundation for subsequent growth stages.

The growth of lateral and thin veins is managed through a polar coordinate system, allowing the model to efficiently "emit" rays to find boundaries. A Global Incentive Loss ($L_g$) is utilized to normalize errors across different scales, ensuring the model pays equal attention to both small and large text instances.

Detailed algorithmic steps for the growth and rectification of lateral and thin veins using polar coordinates.

Experimental Results

LeafText was evaluated on four major benchmarks after pre-training on SynthText:

  • MSRA-TD500: Achieved an 87.8% F-measure, a 1.1% improvement over previous state-of-the-art (SOTA) methods.
  • Total-Text: Reached an 87.3% F-measure, excelling at word-level arbitrary shapes.
  • CTW1500: Hit an 85.5% F-measure, proving effectiveness on long, curved lines.

The one-stage nature of the framework ensures high efficiency by avoiding computationally expensive post-processing steps like complex grouping or Non-Maximum Suppression (NMS).

Conclusion

LeafText demonstrates that bio-inspired geometric structures can effectively solve complex scene text detection problems. By mimicking the hierarchical growth of leaf veins, the model provides a robust solution for irregular text. Future work will address challenges related to false positives in visually similar architectural patterns and text adhesion in densely packed lines.

Summary updated

BibTeX
@article{yang2023text,
  author = {Chuang Yang and Mulin Chen and Yuan Yuan and Qi Wang},
  title = {{Text Growing on Leaf}},
  journal = {IEEE Trans. Multimedia},
  year = {2023},
  url = {https://openreview.net/forum?id=A0eLziUS6E},
}
Figure