Guided Table Structure Recognition through Anchor Optimization

Khurram Azeem Hashmi, Didier Stricker, Marcus Liwicki, Muhammad Noman Afzal

Abstract

This paper presents a novel approach to table structure recognition by leveraging guided anchors. The concept differs from current state-of-the-art approaches that naively apply object detection methods. In contrast to prior techniques, we first estimate viable anchors specifically for table structure recognition and subsequently exploit these anchors to locate rows and columns in tabular images. Furthermore, the paper introduces a simple and effective method that improves results by using tabular layouts in realistic scenarios. The proposed method is exhaustively evaluated on two publicly available datasets: ICDAR-2013 and TabStructDB. We accomplished state-of-the-art results on the ICDAR-2013 dataset with an average F-Measure of 95.05% (94.6% for rows and 96.32% for columns) and surpassed baseline results on the TabStructDB dataset with an average F-Measure of 94.17% (94.08% for rows and 95.06% for columns).

Introduction

Extracting information from tables in scanned documents is a notoriously difficult task in document analysis. While humans can easily interpret a table, machines struggle due to the immense diversity in tabular patterns, such as varying use of ruling lines versus whitespace for alignment.

Table analysis involves two primary sub-problems: table detection (finding the boundary) and table structure recognition (identifying internal rows, columns, and cells). Most existing methods rely on PDF metadata, which fails when dealing with raw, scanned images. This paper focuses on recognizing the internal structure directly from images by treating row and column detection as a specialized object detection problem.

Methodology

The researchers built their system on the Mask R-CNN architecture, utilizing a ResNet-101 backbone and a Feature Pyramid Network (FPN) to extract multi-scale features. They introduced two key innovations to move beyond a "naive" application of object detection:

Anchor Optimization

In standard object detection, anchors (predefined bounding boxes) are typically square-ish. However, tables consist of very wide, thin rows and tall, narrow columns. The authors used K-means clustering on training data to calculate optimal aspect ratios (e.g., 50:1 for rows; 1:10 for columns). This allows the Region Proposal Network (RPN) to capture tabular structures more effectively, leading to faster convergence and higher accuracy.

Layout-Based Post-Processing

Deep learning models often struggle with pixel-perfect boundaries. The authors developed a heuristic algorithm that adjusts predicted row boundaries based on the presence of actual text (black pixels). This ensures that detected rows tightly encapsulate the data, correcting for ambiguous whitespace.

Visual demonstration of row (red) and column (blue) detection across various table layouts using the proposed method.

Experimental Results

The proposed method outperformed previous state-of-the-art models like DeepDeSRT and TableNet. On the ICDAR-2013 benchmark, it achieved an average F-Measure of 95.05%. On the more complex TabStructDB dataset, it reached 94.17%.

A key insight from the experiments was the impact of anchor optimization: by simply adjusting aspect ratios to match tabular geometry, the F-measure for row detection jumped from 0.87 to 0.92. Furthermore, the optimized network reached a low loss value in nearly half the training time (30 epochs vs. 50+) compared to standard models.

Conclusion

This work demonstrates that guiding object detection models with domain-specific priors—specifically optimized anchors—significantly improves table structure recognition. While the system shows high performance, future work will address challenges such as multi-line rows and explore direct cell detection. The authors also suggest that incorporating Coordinate Convolutions (CoordConv) could further improve spatial awareness in document image analysis.

Summary updated

Abstract

This paper presents the novel approach towards table structure recognition by leveraging the guided anchors. The concept differs from current state-of-the-art approaches for table structure recognition that naively apply object detection methods. In contrast to prior techniques, first, we estimate the viable anchors for table structure recognition. Subsequently, these anchors are exploited to locate the rows and columns in tabular images. Furthermore, the paper introduces a simple and effective method that improves the results by using tabular layouts in realistic scenarios. The proposed method is exhaustively evaluated on the two publicly available datasets of table structure recognition i.e ICDAR-2013 and TabStructDB. We accomplished state-of-the-art results on the ICDAR-2013 dataset with an average F-Measure of 95.05$\%$ (94.6$\%$ for rows and 96.32$\%$ for columns) and surpassed the baseline results on the TabStructDB dataset with an average F-Measure of 94.17$\%$ (94.08$\%$ for rows and 95.06$\%$ for columns).
BibTeX
@inproceedings{hashmi2021guided,
  author = {Khurram Azeem Hashmi and Didier Stricker and Marcus Liwicki and Muhammad Noman Afzal and Muhammad Zeshan Afzal},
  title = {{Guided Table Structure Recognition through Anchor Optimization}},
  booktitle = {IEEE Access},
  year = {2021},
  url = {https://openreview.net/forum?id=GZhjqa6jGcW},
}
Figure