TRMER: Transformer-Based End to End Printed Mathematical Expression Recognition
Lists
Only you see this
Abstract
As a fundamental task of transcribing formula images into structural mathematical expressions, Printed Mathematical Expression Recognition (PMER) is wildly used in many fields. However, there is still a lack of an end-to-end approach toward fully exploring the spatial structure and semantic information in the formula to achieve high recognition accuracy. In this work, a Transformer-based Mathematical Expression Recognition (TRMER) model, is proposed to enhance the recognition accuracy. A Dual-Branch Encoder (DBE) is developed to extract multi-scaled feature maps from a formula image so that the spatial and semantic information can be obtained synchronously, and the different feature maps are fused with a Fusion Enhancement Module (FEM) by merging and reinforcing the spatial-semantic information. A standard transformer-based decoder is developed to decode the rich spatial-semantic information of the image and output a recognized mathematical expression in LaTex sequence. The experimental results have illustrated that the TRMER has achieved state-of-the-art recognition performance.
BibTeX
@inproceedings{zhou2023trmer,
author = {Zhaokun Zhou and Shuaijian Ji and Yuqing Wang and Zhenyu Weng and Yuesheng Zhu},
title = {{TRMER: Transformer-Based End to End Printed Mathematical Expression Recognition}},
booktitle = {IJCNN},
year = {2023},
url = {https://openreview.net/forum?id=19AtIExNbA},
}Loading related content...