1st Place Solution for ICDAR 2021 Competition on Mathematical Formula Detection

Yuxiang Zhong, Xianbiao Qi, Shanjun Li, Dengyi Gu, Yihao Chen, Peiyang Ning, Rong XiaoarXiv 2021 · arXiv

Abstract

Mathematical Formula Detection (MFD) is a foundational step in document digitization, yet it remains challenging due to extreme scale variance and complex symbol arrangements. This paper details the winning solution for the ICDAR 2021 MFD competition, which achieved a top-ranking F1-score of 96.33. By transitioning from traditional anchor-based methods to an anchor-free framework utilizing Generalized Focal Loss (GFL) and a custom-tuned Feature Pyramid Network (FPN), the proposed approach significantly improves the detection of tiny embedded formulas while maintaining high accuracy for large isolated expressions.

Challenges in Mathematical Formula Detection

Standard OCR engines often struggle with mathematical expressions because they do not follow the linear, uniform structure of plain text. The authors identify three primary hurdles in MFD:

  1. Extreme Scale Variance: Formulas appear as either "isolated" (large, standalone blocks) or "embedded" (tiny expressions within a line of text). Isolated formulas can be hundreds of times larger than their embedded counterparts.
  2. Aspect Ratio Diversity: Expressions range from vertically tall (integrals, fractions) to horizontally expansive (long equations), making predefined bounding box shapes (anchors) ineffective.
  3. Character Complexity: The dense spatial arrangement of subscripts, superscripts, and unique mathematical symbols complicates feature extraction.
Figure 2: Example of the MFD task showing the vast difference in scale between isolated blocks and tiny embedded formulas.

Proposed Methodology

The core of the solution is an anchor-free framework based on Generalized Focal Loss (GFL). Unlike models like Faster R-CNN that rely on fixed anchors, this approach regresses boundaries directly, making it more robust to the diverse aspect ratios found in mathematics.

Key Technical Innovations

  • High-Resolution Feature Pyramids (P2–P6): Standard FPNs typically use levels P3–P7. However, tiny embedded formulas (often ~16 pixels high) are often lost at these resolutions. By shifting the network to use levels P2–P6, the model can detect much smaller features.
  • Adaptive Training Sample Selection (ATSS): To prevent the model from becoming biased toward large formulas, ATSS ensures that every instance receives an equal number of positive training samples regardless of its physical size.
  • Advanced Backbone and Optimizer: The team utilized ResNeSt101 for feature extraction and the Ranger optimizer (combining RAdam and LookAhead) for more stable convergence.
  • Deformable Convolutional Networks (DCN): DCN allows the network's receptive field to adapt to the specific, non-rectangular shapes of complex equations.
  • Weighted Box Fusion (WBF): Instead of simple Non-Maximum Suppression (NMS), WBF blends overlapping predictions to produce a more accurate final bounding box.

Experimental Results

The model was trained and evaluated on the IBEM dataset, which contains over 300,000 expressions across 8,273 pages. The team (PAPCIC) secured 1st place out of 15 competing teams.

Category F1-Score
Total 96.33%
Isolated Formulas 98.14%
Embedded Formulas 96.01%
Figure 5: Detection results on a complex block of isolated equations, demonstrating the model's precision in handling multi-line mathematical structures.

The results highlight that while most competitors performed well on isolated formulas, the PAPCIC team’s primary advantage was their superior performance on embedded formulas, where they outperformed lower-ranked teams by more than 10%.

Conclusion

The success of this solution underscores the importance of high-resolution feature maps (the "P2 breakthrough") and scale-invariant training strategies like ATSS in document analysis. While general object detection "tricks" like CutMix or GIoU loss did not provide benefits in this specific domain, the shift to anchor-free architectures proved decisive. Future work will focus on integrating these high-quality detections into full recognition pipelines to convert images directly into LaTeX or MathML.

Summary updated

Abstract

In this technical report, we present our 1st place solution for the ICDAR 2021 competition on mathematical formula detection (MFD). The MFD task has three key challenges including a large scale span, large variation of the ratio between height and width, and rich character set and mathematical expressions. Considering these challenges, we used Generalized Focal Loss (GFL), an anchor-free method, instead of the anchor-based method, and prove the Adaptive Training Sampling Strategy (ATSS) and proper Feature Pyramid Network (FPN) can well solve the important issue of scale variation. Meanwhile, we also found some tricks, e.g., Deformable Convolution Network (DCN), SyncBN, and Weighted Box Fusion (WBF), were effective in MFD task. Our proposed method ranked 1st in the final 15 teams.
BibTeX
@misc{zhong2021st,
  author = {Yuxiang Zhong and Xianbiao Qi and Shanjun Li and Dengyi Gu and Yihao Chen and Peiyang Ning and Rong Xiao},
  title = {{1st Place Solution for ICDAR 2021 Competition on Mathematical Formula Detection}},
  year = {2021},
  eprint = {2107.05534},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2107.05534},
}
Figure