Text Growing on Leaf

Chuang Yang, Mulin Chen, Yuan Yuan, Qi Wang

TL;DR

LeafText is a bio-inspired scene text detection framework that represents irregular text contours as "leaf margins" supported by a hierarchical structure of main, lateral, and thin veins. By simulating the growth process of a leaf, the model achieves state-of-the-art accuracy in fitting highly curved, ribbon-like text lines that traditional bounding-box or simple regression methods often miss.

What Problem Does This Paper Address?

Scene Text Detection (STD) is the task of identifying and localizing text in natural images—a critical step for autonomous systems and translation apps. While deep learning has mastered "regular" (rectangular) text, "irregular" text—curved, slanted, or wrapped around objects—remains a major hurdle.

Existing approaches generally fall into two camps: segmentation-based (which requires massive amounts of pixel-level data) and regression-based (which predicts point sequences). Regression methods often struggle with "ribbon-like" text because multiple contour points can end up in the same direction, leading to messy or overlapping results. Furthermore, many models fail to balance "multi-scale" text, often ignoring small text instances in favor of larger, easier-to-detect ones.

Methods

The authors introduce LeafText, a one-stage framework built on a novel Leaf Vein-based Text representation (LVT). The core idea is to treat the text contour as a leaf margin and reconstruct it through a three-tiered "vein" hierarchy:

  1. Main Vein (MV): Extracted from a predicted kernel mask, this acts as the "spine" of the text instance, providing a rough location and orientation.
  2. Lateral Veins (LV): These sprout perpendicularly from the main vein to determine the coarse boundaries of the text.
  3. Thin Veins (TV): These are shorter refinements that sprout from the lateral veins to "fine-tune" the final contour, ensuring high precision even for jagged or highly curved edges.

To make this work, the authors developed two key components:

  • Multi-Oriented Smoother (MOS): Standard sampling often results in "jagged" center lines. MOS uses a rotating coordinate system and polynomial fitting to ensure the "main vein" is smooth and robust, providing a reliable foundation for the lateral veins.
  • Global Incentive Loss ($L_g$): Instead of standard L2 loss, they use a negative logarithm loss that normalizes errors across different scales. This "incentivizes" the model to pay equal attention to tiny text and massive billboards.

Datasets

The model was pre-trained on SynthText (800k synthetic images) and evaluated on four major benchmarks:

  • MSRA-TD500: Focuses on multi-oriented, line-level Chinese and English text.
  • CTW1500: Contains 1,500 samples of line-level curved text.
  • Total-Text: A dataset specifically for word-level arbitrary-shaped and curved text.
  • ICDAR2015: A standard "incidental" scene text dataset with complex backgrounds.

Key Insights and Ideas

  • Bionics as Geometry: The "aha moment" here is realizing that a leaf's vein structure is essentially a directed graph that naturally covers irregular geometries. By mimicking this, the authors solve the "directionality" problem that plagues other regression models.
  • Polar vs. Cartesian: The model grows the main vein in Cartesian coordinates (standard x, y) but generates lateral and thin veins in polar coordinates. This allows the model to "emit" rays outward to find the text boundary more efficiently.
  • The Power of "Thin Veins": The authors found that adding the "thin vein" layer—essentially a second level of refinement—reduced model complexity while accelerating training convergence. It’s a "divide and conquer" strategy for precision.

Key Results

LeafText outperformed existing state-of-the-art (SOTA) methods across the board:

  • MSRA-TD500: Achieved an 87.8% F-measure, surpassing the previous best (ReLaText) by 1.1%.
  • Total-Text: Reached 87.3% F-measure, a significant jump over traditional segmentation-based models.
  • CTW1500: Hit 85.5% F-measure, proving its effectiveness on long, curved text lines.
  • Efficiency: Because it is a one-stage framework, it avoids the computationally expensive post-processing (like complex grouping or NMS) required by many multi-stage detectors.

Limitations and Future Work

Despite its success, the authors note two primary challenges:

  1. False Positives: The model occasionally identifies non-text regions with high visual similarity to text (e.g., certain architectural patterns) as text instances.
  2. Text Adhesion: When two text lines are extremely close to each other, the "veins" can sometimes overlap or "over-emit," causing the model to struggle to distinguish between separate instances.

Future work will focus on improving the model's ability to distinguish between these interference regions and refining the "growth" process to prevent instance overlap.

Summary updated

Abstract

Irregular-shaped texts bring challenges to Scene Text Detection (STD). Although existing regression-based approaches achieve comparable performances, they fail to cover some highly curved ribbon-like text lines. Inspired by morphology, we found that the leaf vein can easily cover various geometries. Specifically, lateral and thin veins are emitted to margin along main vein gradually with the leaf growth. This process can decompose a concave object into consecutive convex regions, which are easier to fit. Hence, the leaf vein is suitable for representing highly curved texts. Considering the aforementioned advantage, we design a leaf vein-based text representation method (LVT), where text contour is treated as leaf margin and represented through main, lateral, and thin veins. We further construct a detection framework based on LVT, namely LeafText. In the text reconstruction stage, LeafText simulates the leaf growth process to rebuild text contours. It grows main veins in Cartesian coordinates to locate texts roughly at first. Then, lateral and thin veins are generated along the main vein growth direction in polar coordinates. They are responsible for generating the coarse contour and refining it, respectively. Meanwhile, Multi-Oriented Smoother (MOS) is designed to smooth the main vein for ensuring reliable growth directions of lateral and thin veins. Additionally, a global incentive loss is proposed to enhance the predictions of lateral and thin veins. Ablation experiments demonstrate LVT can fit irregular-shaped texts precisely and verify the effectiveness of MOS and global incentive loss. Comparisons show that LeafText is superior to existing state-of-the-art (SOTA) methods on MSRA-TD500, CTW1500, Total-Text, and ICDAR2015 datasets.
BibTeX
@article{yang2023text,
  author = {Chuang Yang and Mulin Chen and Yuan Yuan and Qi Wang},
  title = {{Text Growing on Leaf}},
  journal = {IEEE Trans. Multimedia},
  year = {2023},
  url = {https://openreview.net/forum?id=A0eLziUS6E},
}
Figure