Abstract
Mathematical Formula Detection (MFD) is a foundational step in document digitization, yet it remains challenging due to extreme scale variance and complex symbol arrangements. This paper details the winning solution for the ICDAR 2021 MFD competition, which achieved a top-ranking F1-score of 96.33. By transitioning from traditional anchor-based methods to an anchor-free framework utilizing Generalized Focal Loss (GFL) and a custom-tuned Feature Pyramid Network (FPN), the proposed approach significantly improves the detection of tiny embedded formulas while maintaining high accuracy for large isolated expressions.
Challenges in Mathematical Formula Detection
Standard OCR engines often struggle with mathematical expressions because they do not follow the linear, uniform structure of plain text. The authors identify three primary hurdles in MFD:
- Extreme Scale Variance: Formulas appear as either "isolated" (large, standalone blocks) or "embedded" (tiny expressions within a line of text). Isolated formulas can be hundreds of times larger than their embedded counterparts.
- Aspect Ratio Diversity: Expressions range from vertically tall (integrals, fractions) to horizontally expansive (long equations), making predefined bounding box shapes (anchors) ineffective.
- Character Complexity: The dense spatial arrangement of subscripts, superscripts, and unique mathematical symbols complicates feature extraction.
Proposed Methodology
The core of the solution is an anchor-free framework based on Generalized Focal Loss (GFL). Unlike models like Faster R-CNN that rely on fixed anchors, this approach regresses boundaries directly, making it more robust to the diverse aspect ratios found in mathematics.
Key Technical Innovations
- High-Resolution Feature Pyramids (P2–P6): Standard FPNs typically use levels P3–P7. However, tiny embedded formulas (often ~16 pixels high) are often lost at these resolutions. By shifting the network to use levels P2–P6, the model can detect much smaller features.
- Adaptive Training Sample Selection (ATSS): To prevent the model from becoming biased toward large formulas, ATSS ensures that every instance receives an equal number of positive training samples regardless of its physical size.
- Advanced Backbone and Optimizer: The team utilized ResNeSt101 for feature extraction and the Ranger optimizer (combining RAdam and LookAhead) for more stable convergence.
- Deformable Convolutional Networks (DCN): DCN allows the network's receptive field to adapt to the specific, non-rectangular shapes of complex equations.
- Weighted Box Fusion (WBF): Instead of simple Non-Maximum Suppression (NMS), WBF blends overlapping predictions to produce a more accurate final bounding box.
Experimental Results
The model was trained and evaluated on the IBEM dataset, which contains over 300,000 expressions across 8,273 pages. The team (PAPCIC) secured 1st place out of 15 competing teams.
| Category | F1-Score |
|---|---|
| Total | 96.33% |
| Isolated Formulas | 98.14% |
| Embedded Formulas | 96.01% |
The results highlight that while most competitors performed well on isolated formulas, the PAPCIC team’s primary advantage was their superior performance on embedded formulas, where they outperformed lower-ranked teams by more than 10%.
Conclusion
The success of this solution underscores the importance of high-resolution feature maps (the "P2 breakthrough") and scale-invariant training strategies like ATSS in document analysis. While general object detection "tricks" like CutMix or GIoU loss did not provide benefits in this specific domain, the shift to anchor-free architectures proved decisive. Future work will focus on integrating these high-quality detections into full recognition pipelines to convert images directly into LaTeX or MathML.
Summary updated