Flexible Geometric Modeling for Arbitrary Shaped Text Detection

379 concepts199 papers1 proposal

Traditional object detection relies on axis-aligned or oriented rectangular bounding boxes. While these work for rigid objects like cars or faces, they fail for scene text, which frequently appears on curved surfaces, crumpled packaging, or artistic signage. As noted by Chen et al., rigid boxes often include excessive background noise or fail to encompass the full text area, which complicates downstream recognition.

To address this, researchers have moved toward flexible geometric representations that can capture the "ribbon-like" nature of text.

From Pixels to Polygons

The most direct way to handle arbitrary shapes is through semantic or instance segmentation. By predicting text at the pixel level, models can naturally accommodate any curvature without predefined geometric priors.

However, raw segmentation masks can be noisy and difficult to separate when text instances are close together. Kobchaisawat et al. addressed this by using polygon offsetting, generating inward and outward border masks to effectively distinguish adjacent text instances. Similarly, Zhu et al. proposed TextMountain, which predicts a "center-border probability" map. This creates a gradient—much like a mountain peak—that allows the model to separate individual text lines even in dense, complex layouts.

Parametric Curve Fitting

While segmentation is flexible, it is often computationally expensive and requires complex post-processing to convert pixels back into usable coordinates. A more efficient alternative is to treat text boundaries as mathematical curves.

  • Bezier Curves: Shen et al. and the authors of ABCNet proposed representing text boundaries using parameterized Bezier curves. By introducing a "BezierAlign" layer, the model can sample features directly along the curved path of the text, enabling end-to-end spotting that is significantly faster than segmentation-based methods.
  • Polynomials and Contours: Other methods use different mathematical bases. Wang et al. introduced TextRay, which models contours in a polar coordinate system using Chebyshev polynomials. Liu et al. proposed the Polynomial Band (PB) representation, which captures complex shapes with a fixed number of parameters, avoiding the inefficiency of variable point-based methods.

Frequency Domain and Low-Rank Representations

Recent innovations have moved beyond spatial coordinates entirely, looking for more compact ways to describe shape. The intuition is that text shapes are not random; they possess structural correlations, such as parallel top and bottom edges.

Su et al. proposed TextDCT, which uses the Discrete Cosine Transform (DCT) to encode text masks into compact frequency-domain vectors. This allows the model to regress a smooth, high-quality mask using only a few coefficients.

Building on the idea of structural patterns, Su et al. also developed LRANet, which uses Low-Rank Approximation (LRA). By learning "eigenanchors" from training data, the model can reconstruct complex, arbitrary shapes using significantly fewer parameters than standard regression. This approach exploits the fact that most text shapes share a common underlying geometry, making the optimization task much simpler for the neural network.

Skeleton and Point-Based Modeling

Another perspective focuses on the "topology" of the text rather than just its outer boundary. Xue et al. argued that learning the internal skeleton of text through keypoints and links allows models to generalize to warped shapes even when trained on straight text. By connecting central keypoints with horizontal and vertical links, the model builds a flexible framework that can stretch and curve to fit the instance.

Similarly, Long et al. proposed Hierarchical Point Modeling (HIP), which treats document elements as a hierarchy of points. This moves away from the "box" paradigm entirely, allowing for a unified treatment of characters, words, and entities in complex, non-linear layouts.

History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_arbitrary_shaped_text_detection_e76eacf8,
  title = {arbitrary-shaped text detection},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/arbitrary-shaped-text-detection/rdr_e76eacf84ad94925b1ac8a3bab904f87},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-22},
}

This exact version (version 1, frozen text):

@misc{lacuna_arbitrary_shaped_text_detection_e76eacf8_v1,
  title = {arbitrary-shaped text detection},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/arbitrary-shaped-text-detection/rdr_e76eacf84ad94925b1ac8a3bab904f87/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-22},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake