TL;DR
This paper introduces a specialized approach to table structure recognition by treating row and column detection as an object detection problem. By optimizing the "anchors" used in a Mask R-CNN framework to match the extreme aspect ratios of tabular data and applying a layout-based post-processing step, the authors achieved new state-of-the-art results on the ICDAR-2013 and TabStructDB benchmarks.
What Problem Does This Paper Address?
Extracting information from tables in scanned documents is a notoriously difficult task in document analysis. While humans can easily read a table, machines struggle because of the immense diversity in tabular patterns: some have clear ruling lines, others use white space for alignment, and many share visual similarities with charts or figures.
The authors identify two sub-problems: table detection (finding the table's boundary) and table structure recognition (identifying the internal rows, columns, and cells). Most existing methods rely on PDF metadata (which contains hidden text and coordinate data), but these fail when dealing with raw, scanned images. This paper focuses on recognizing the internal structure of tables directly from images, specifically targeting the accurate detection of rows and columns.
Methods
The researchers built their system on the Mask R-CNN architecture, a popular deep learning model for object detection and instance segmentation. However, they moved away from the "naive" application of this model by introducing two key innovations:
- Anchor Optimization: In standard object detection (like identifying dogs or cars), "anchors" (predefined bounding boxes) are usually square-ish. Tables, however, consist of very wide, thin rows and very tall, narrow columns. The authors used K-means clustering on the training data to calculate optimal aspect ratios for these anchors. For rows, they used ratios like 50:1; for columns, ratios like 1:10. This allows the Region Proposal Network (RPN) to "see" tabular structures more effectively and converge faster during training.
- Layout-Based Post-Processing: Deep learning models often struggle to find the exact pixel-perfect boundary of a row, sometimes leaving extra white space. The authors developed a heuristic algorithm that "shrinks" or "expands" the predicted row boundaries based on the presence of black pixels (actual text). This ensures the detected rows tightly encapsulate the data.
The backbone of the model uses a combination of ResNet-101 and a Feature Pyramid Network (FPN) to extract multi-scale features from the document images.
Datasets
The model was trained and evaluated on two primary datasets:
- ICDAR-2013: A standard benchmark for table competition. The authors used a modified version that provides image-based annotations for rows and columns.
- TabStructDB: A more recent and complex dataset derived from the ICDAR-2017 Page Object Detection (POD) competition, containing 731 tabular regions for training and 350 for testing.
Key Insights and Ideas
- Anchors Matter: The "aha moment" here is that generic object detection settings are poorly suited for the geometry of tables. By simply adjusting the anchor aspect ratios to match the "long and thin" nature of rows and columns, the model's F-measure for row detection jumped from 0.87 to 0.92.
- Faster Convergence: Optimized anchors don't just improve accuracy; they make training more efficient. The network with optimized anchors reached a low loss value (under 0.1) in nearly half the time (30 epochs vs. 50+) compared to the standard model.
- Hybrid Approach: While deep learning handles the heavy lifting of detection, simple "old school" computer vision heuristics (like scanning for black pixels) are still highly effective for fine-tuning boundaries that neural networks might find ambiguous.
Key Results
The proposed method outperformed previous state-of-the-art models like DeepDeSRT and TableNet:
- ICDAR-2013: Achieved an average F-Measure of 95.05% (94.6% for rows and 96.32% for columns).
- TabStructDB: Achieved an average F-Measure of 94.17%, surpassing the previous baseline.
- Cross-Dataset Generalization: When trained on one dataset and tested on the other, the model maintained a respectable F-measure (approx. 0.74–0.77), highlighting the difficulty of generalizing across different annotation schemes (e.g., some datasets annotate only text regions, while others annotate the full width of the table).
Limitations and Future Work
Despite the improvements, the model still faces challenges:
- Extreme Geometry: It occasionally fails to detect very narrow rows or columns.
- Multi-line Rows: The system sometimes struggles with rows where text spans multiple lines, occasionally splitting them into two separate rows.
- Direct Cell Detection: The authors suggest that instead of detecting rows and columns and then intersecting them to find cells, future work could explore detecting cells directly. They also mention that incorporating CoordConv (Coordinate Convolution) might provide the network with better spatial awareness of where it is looking within a document.
Summary updated