AttentionHTR: Handwritten Text Recognition Based on Attention Encoder-Decoder Networks

Dmitrijs Kass, Ekta Vats

Abstract

This work proposes an attention-based sequence-to-sequence model for handwritten word recognition and explores transfer learning for data-efficient training of HTR systems. To overcome training data scarcity, this work leverages models pre-trained on scene text images as a starting point towards tailoring the handwriting recognition models. ResNet feature extraction and bidirectional LSTM-based sequence modeling stages together form an encoder. The prediction stage consists of a decoder and a content-based attention mechanism. The effectiveness of the proposed end-to-end HTR system has been empirically evaluated on a novel multi-writer dataset Imgur5K and the IAM dataset. The experimental results evaluate the performance of the HTR framework, further supported by an in-depth analysis of the error cases. Source code and pre-trained models are available at GitHub (https://github.com/dmitrijsk/AttentionHTR).
BibTeX
@inproceedings{kass2022attentionhtr,
  author = {Dmitrijs Kass and Ekta Vats},
  title = {{AttentionHTR: Handwritten Text Recognition Based on Attention Encoder-Decoder Networks}},
  booktitle = {DAS},
  year = {2022},
  doi = {10.1007/978-3-031-06555-2\_34},
}

Loading related content...

Figure