Linecounter: Learning Handwritten Text Line Segmentation By Counting

Deng Li, Yue Wu, Yicong Zhou

TL;DR

This paper introduces "Line Counting," a novel formulation for handwritten text line segmentation (HTLS) that directly predicts the line number for each pixel. They propose a new end-to-end deep learning model, LineCounter, which leverages this formulation and a unique "cumsum" activation to achieve state-of-the-art results on multiple datasets without complex post-processing.

What Problem Does This Paper Address?

Handwritten Text Line Segmentation (HTLS) is a crucial preprocessing step for many higher-level document analysis tasks, such as handwritten text recognition. Existing deep learning approaches often adapt methods from semantic segmentation or object detection. However, these methods have significant drawbacks: semantic segmentation requires cumbersome post-processing to merge or split predicted segments, while object detection can struggle with dense or curved text lines. Furthermore, these adapted methods often necessitate converting the ground truth HTLS line maps into formats suitable for their respective tasks, and then converting the model's predictions back, introducing ambiguity and potential errors. This paper aims to address these limitations by proposing a new formulation and model specifically designed for HTLS.

Methods

The core innovation is the Line Counting formulation. Instead of predicting masks or bounding boxes, the model predicts, for each pixel, the number of text lines above it. This is framed as a per-pixel regression task. The authors propose a deep neural network called LineCounter, built on an Encoder-Decoder architecture. A key component is the Counter module, which uses GRU layers to propagate line information both horizontally and vertically, enabling the network to understand the spatial context of lines.

A critical aspect of the Line Counting formulation is the desire for monotonicity: line numbers should not decrease as you move down the page. To enforce this, the authors introduce a novel cumsum activation. This activation, applied to feature maps, inherently enforces a non-decreasing property when the input is non-negative. They found that applying this cumsum activation before the final decoder layer, combined with a hard sigmoid to ensure non-negativity, yields the best results. This approach allows for end-to-end training directly from HTLS line map annotations without the need for intermediate conversions or complex post-processing steps like voting or merging.

Datasets

The paper evaluates LineCounter on three publicly available datasets:

  • ICDAR2013-HSC: Contains Latin and Bangla text.
  • HIT-MW: Contains Chinese text.
  • VML-AHTE: Contains Arabic text.

The authors note that the VML-AHTE dataset has denser text lines and a very small training set, making it particularly challenging.

Key Insights and Ideas

  • Rethinking the HTLS formulation: Framing HTLS as a "counting" problem, rather than segmentation or detection, is a novel and effective approach. It directly addresses the inherent structure of text lines on a page.
  • The power of monotonicity: Enforcing the monotonic increase of line numbers is crucial for accurate HTLS and can be achieved elegantly with a modified cumsum function. This constraint implicitly helps the model learn the semantics of non-text regions.
  • End-to-end simplicity: The Line Counting formulation and LineCounter model eliminate the need for complex, error-prone post-processing steps that plague previous methods. This leads to a cleaner and more efficient pipeline.
  • Counter module's importance: The specialized Counter module, with its bidirectional GRUs, is essential for propagating line information and understanding spatial relationships, which is vital for accurate counting.

Key Results

LineCounter achieved state-of-the-art performance on all three evaluated datasets, outperforming previous methods in terms of Detection Rate (DR), Recognition Accuracy (RA), and F-Measure (FM). Notably, it achieved the best F-Measure scores, often by 1-2% over the next best approach. The model is also computationally efficient, achieving 2.5 FPS on an Nvidia 2080Ti GPU with a relatively light architecture (13M parameters). The cumsum activation significantly improved performance, boosting the F-Measure from 0.854 to 0.973 on a baseline model.

Limitations and Future Work

The authors acknowledge that while LineCounter has outperformed existing methods, the architecture and hyper-parameters have not been fully optimized. They suggest that similar "counting" formulations could be beneficial for related tasks like word segmentation and paragraph segmentation. Future work could involve further architectural refinements and hyper-parameter tuning to potentially push performance even higher.

Summary updated

BibTeX
@inproceedings{li2021linecounter,
  author = {Deng Li and Yue Wu and Yicong Zhou},
  title = {{Linecounter: Learning Handwritten Text Line Segmentation By Counting}},
  booktitle = {ICIP},
  year = {2021},
  doi = {10.1109/icip42928.2021.9506664},
}

Loading related content...

Figure