Hierarchical Multi-label Learning for Incremental Multilingual Text Recognition

Xiao-Qian Liu, Minghui Liu, Zhen-Duo Chen, Xin Luo, Xin-Shun Xu

Abstract

Multilingual text recognition (MLTR) is increasingly essential for facilitating cultural communication. However, existing methods often struggle with retaining previous language knowledge when learning new languages. A straightforward solution is performing incremental learning (IL) on MLTR tasks. However, it ignores the shared words and characters across incremental languages, which we first term as an incremental sharing problem. Motivated by this observation, we propose a HierArchical Multi-label learning framework for Multilingual tExt Recognition, termed HAMMER. An online knowledge analysis is designed to identify shared knowledge and provide corresponding multi-label language supervision. Specifically, only words and characters appearing simultaneously in multiple languages are considered shared knowledge. Additionally, to further capture language dependencies, we introduce a hierarchical language evaluation mechanism to predict language scores at word and character levels. These scores, supervised by the knowledge analysis, guide the specific recognizers to effectively utilize both old and new language knowledge, thereby mitigating catastrophic forgetting caused by imbalanced rehearsal sets. Extensive experiments conducted on benchmark datasets, MLT17 and MLT19, show that HAMMER exhibits remarkable results and outperforms other state-of-the-art approaches.
BibTeX
@inproceedings{liu2024hierarchical,
  author = {Xiao-Qian Liu and Minghui Liu and Zhen-Duo Chen and Xin Luo and Xin-Shun Xu},
  title = {{Hierarchical Multi-label Learning for Incremental Multilingual Text Recognition}},
  booktitle = {ACM Multimedia},
  year = {2024},
  doi = {10.1145/3664647.3681350},
}

Loading related content...

Figure