Abstract
The WeLayout system addresses the challenge of robust document layout analysis (DLA) in complex corporate documents. By combining DINO and YOLOv8 architectures with a novel "cell matching" refinement process that leverages PDF metadata, the system achieves high precision in identifying document elements like paragraphs and tables. The approach utilizes a multi-stage pipeline including document classification and an ensemble strategy optimized through Bayesian search. WeLayout secured first place in the ICDAR 2023 Competition with an mAP of 70.0, significantly outperforming competitors in handling unseen document categories.
Introduction
Automated document processing relies heavily on Document Layout Analysis (DLA)—the ability to categorize regions of a page such as paragraphs, tables, or headers. While standard OCR works well for plain text, corporate documents like financial reports, patents, and contracts feature complex, multi-column layouts and diverse visual elements that often confuse traditional systems. The ICDAR 2023 competition challenged researchers to build models robust across different document styles, specifically highlighting the difficulty of categories like "Others" which were not explicitly labeled in the training data.
Methods
The WeLayout system is a multi-stage pipeline designed for maximum precision, integrating classification, detection, and refinement stages.
1. Document Classification
Since different document types (e.g., patents vs. financial reports) possess distinct visual "grammars," the system uses a Vision Transformer (ViT) to classify incoming pages. This allows for category-specific fine-tuning, which the authors found significantly improved performance. For instance, YOLOv8's performance on manuals increased from 61.0 to 63.0 mAP when trained exclusively on that category.
2. Model Ensemble
The team combined two complementary state-of-the-art architectures:
- DINO (Query-Based): A transformer-based detector using learnable queries to locate objects.
- YOLOv8 (Anchor-Free): A high-speed detector that predicts objects per-pixel, avoiding the limitations of predefined anchor boxes.
3. Bounding Box Refinement (Cell Matching)
A standout feature of WeLayout is the use of the PDF text layer. Most digital PDFs contain "text cells" (coordinates of words/characters). The authors developed an algorithm to "snap" predicted bounding boxes to these precise boundaries. This post-processing step boosted DINO’s performance by 9.2 mAP points, demonstrating that deep learning models benefit greatly from ground-truth structural hints.
4. Weighted Box Fusion (WBF)
To merge predictions from DINO and YOLO, the system employs Weighted Box Fusion. The fusion parameters (weights and overlap thresholds) were optimized using Bayesian optimization (Tree-Structured Parzen Estimator), searching a hyperparameter space of nearly $10^{12}$ possibilities.
Datasets and Augmentation
The primary training data was DocLayNet (80,000+ annotated pages). To improve robustness, the authors harvested tables and figures from external datasets like IIIT-AR-13K and TNCR. Crucially, they generated 300,000 synthetic images by cropping elements and pasting them onto new backgrounds with varying column layouts.
Key Results
WeLayout achieved 1st place in the competition with an official mAP of 70.0. A key strength was its generalization; it outperformed the second-place team by 10 mAP points in the "Others" category. On an internal combined validation/test set, the full pipeline reached a peak performance of 91.0 mAP.
Conclusion
The success of WeLayout underscores the importance of combining diverse model architectures and leveraging existing metadata like PDF text layers. While the "Others" category remains a challenge due to the lack of dedicated training data, the system's classification-based approach provides a strong foundation for future robust layout analysis across even broader document types.
Summary updated