Geometric Representations for Arbitrary Shape Text Detection

This direction is merged into another direction: arbitrary-shaped text detection. Its last version stays readable and citable here.

Version 1 of 161 papers63 conceptsfrozen 2026-10-04

In the world of computer vision, most objects are "compact." A car, a person, or a chair generally fits neatly into a rectangular bounding box. However, as Shangbang Long and Cong Yao observe in Scene Text Detection and Recognition: The Deep Learning Era, text in the wild is fundamentally different. It appears on curved neon signs, wraps around coffee cups, and stretches across long banners with extreme aspect ratios.

When we force these irregular shapes into axis-aligned rectangles, two things happen: we include too much irrelevant background noise, and we lose the "flow" of the characters. This makes subsequent recognition—actually reading the words—exponentially harder. To solve this, researchers have moved away from rigid boxes toward flexible geometric representations that can "shrink-wrap" around text of any shape.

The Failure of the Rectangle

Standard object detectors struggle with text because linguistic sequences lack fixed, closed boundaries. As Tong He and colleagues point out in their work on the Connectionist Text Proposal Network (CTPN), a single word can be a square or a thin line 20 times longer than it is tall.

If a detector uses a standard rectangle to capture a curved word, the box must be large enough to encompass the entire arc. This box will inevitably contain parts of other words or complex background textures, confusing the model. To reach "state-of-the-art" accuracy, the field had to rethink the primitive unit of detection.

Segmentation as a Flexible Foundation

One of the most intuitive ways to handle arbitrary shapes is to stop thinking about "boxes" entirely and start thinking about "pixels." In Fused Text Segmentation Networks, Yuchen Dai and others treat detection as an instance-aware semantic segmentation problem.

By predicting a pixel-level mask, the model can follow the exact contour of a curved or tilted word. The Pixel Aggregation Network (PAN) refined this by using a learnable grouping strategy to cluster these pixels into coherent text instances in real-time. The intuition here is simple: if you can identify which pixels belong to a "stroke," you can build the word from the bottom up, regardless of how it is oriented.

Parameterizing the Curve with Bezier Surfaces

While segmentation is flexible, it is often computationally expensive and requires complex post-processing to turn a "cloud of pixels" into a structured shape. ABCNet, introduced by Hao Chen and Yuliang Liu, proposed a more elegant mathematical solution: Bezier curves.

Instead of predicting thousands of pixels, the model regresses a small number of control points that define a smooth curve. This allows the network to "fit" a mathematical spine to the text. By using a "BezierAlign" layer, the model can then "unroll" the curved text into a straight horizontal strip, making it easy for a standard recognition engine to read. This transition from discrete pixels to continuous parameters allows for high-speed detection without sacrificing the ability to handle complex geometries.

Boundary Evolution and Point Queries

The most recent shift involves treating the text boundary as a dynamic entity that can be refined over time. Shi-Xue Zhang and Xiaobin Zhu developed the Adaptive Boundary Proposal Network (TextBPN), which uses Graph Convolutional Networks (GCNs) to iteratively deform a proposal until it perfectly matches the text's edge.

This "coarse-to-fine" approach has been further streamlined by Transformer architectures. In DeepSolo, Jing Zhang and Maoyuan Ye represent text as a sequence of explicit point queries. Rather than looking for a box, the Transformer decoder asks, "Where are the key points along this text's centerline?"

By treating a word as a sequence of points rather than a rigid object, these models can adapt to any degree of curvature or orientation. This unified approach—where detection and recognition happen through the same set of geometric points—represents the current frontier in reading "text in the wild."

Papers in this version

Showing 60 of 61 papers.

History & citationVersion 1 · 2026-10-04

Merged into another direction · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_geometric_representations_for_arbitrary__20e7c2ff,
  title = {Geometric Representations for Arbitrary Shape Text Detection},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/geometric-representations-for-arbitrary-shape-text-detection/rdr_20e7c2ff97c1425e92e392c5aa4ddd2d},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-22},
}

This exact version (version 1, frozen text):

@misc{lacuna_geometric_representations_for_arbitrary__20e7c2ff_v1,
  title = {Geometric Representations for Arbitrary Shape Text Detection},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/geometric-representations-for-arbitrary-shape-text-detection/rdr_20e7c2ff97c1425e92e392c5aa4ddd2d/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-22},
}