Maximum Entropy Regularization and Chinese Text Recognition

Changxu Cheng, Wuheng Xu, Xiang Bai, Bin Feng, Wenyu Liu

Abstract

Chinese text recognition is more challenging than Latin text due to the large amount of fine-grained Chinese characters and the great imbalance over classes, which causes a serious overfitting problem. We propose to apply Maximum Entropy Regularization to regularize the training process, which is to simply add a negative entropy term to the canonical cross-entropy loss without any additional parameters and modification of a model. We theoretically give the convergence probability distribution and analyze how the regularization influence the learning process. Experiments on Chinese character recognition, Chinese text line recognition and fine-grained image classification achieve consistent improvement, proving that the regularization is beneficial to generalization and robustness of a recognition model.

Introduction

Recognizing Chinese text is a "fine-grained" classification challenge. Unlike Latin scripts with a few dozen characters, Chinese involves thousands of unique characters. Many of these characters look nearly identical (high inter-class similarity) while appearing in vastly different handwritten or artistic styles (high intra-class variance).

Fine-grained Chinese characters and the resulting probability distribution from a model.

Because some characters appear much more frequently than others, deep learning models tend to overfit. They become "overconfident," collapsing the output probability distribution into a one-hot vector. This overconfidence causes the model to ignore subtle features that distinguish similar characters and makes it brittle when encountering noisy backgrounds or mislabeled training data.

Maximum Entropy Regularization (MER)

To address this, we introduce Maximum Entropy Regularization (MER). Standard training uses Cross-Entropy (CE) loss, which pushes the model to assign a probability of 1.0 to the correct class. We add a second term to the loss function: $$L_{REG} = L_{CE} + \lambda L_{MER}$$ where $L_{MER}$ is the negative entropy of the predicted distribution.

The "Force" Analogy

The gradient of the regularized loss acts like a physical balance between two competing forces. Cross-entropy ($F_{CE}$) is a force trying to make the model certain (one-hot distribution), while MER ($F_{MER}$) is a force trying to make the model uncertain (uniform distribution).

The gradient of the regularized loss illustrated as a balance of forces between Cross-Entropy and Maximum Entropy Regularization.

When balanced by the hyperparameter $\lambda$, the model converges to a point where it is confident enough to be correct, but "leaves enough room" in the probability space for similar-looking characters. This method requires zero additional parameters and no changes to the model architecture.

Experimental Evaluation

The approach was evaluated on three challenging benchmarks:

  • CTW (Chinese Text in the Wild): Street view images with over 1 million characters across 3,850 classes.
  • ReCTS (Reading Chinese Text on Signboard): Signboard text featuring 4,435 unique characters.
  • CUB-200-2011: A fine-grained bird classification dataset used to prove generalizability beyond OCR.

Key Results

The addition of MER provided consistent gains across the board:

  • CTW Character Recognition: Accuracy improved from 78.53% to 79.24%. Notably, a MER-enhanced ResNet50 outperformed a standard ResNet152.
  • ReCTS Line Recognition: Accuracy rose from 76.89% to 77.50%.
  • Fine-Grained Classification (CUB): Achieved 87.3% accuracy, outperforming several specialized architectures despite using a simpler backbone.
  • Robustness: At a 20% label corruption rate on CUB, the MER model maintained 81.91% accuracy, while the standard model dropped to 79.01%.

Conclusion

Maximum Entropy Regularization effectively mitigates overfitting in fine-grained Chinese text recognition by preventing model overconfidence. By balancing certainty with entropy, the model focuses on more discriminative regions and exhibits higher robustness to noise. Future work may involve incorporating character-structure knowledge directly into the entropy term to further refine distinctions between similar characters.

Summary updated

Abstract

Chinese text recognition is more challenging than Latin text due to the large amount of fine-grained Chinese characters and the great imbalance over classes, which causes a serious overfitting problem. We propose to apply Maximum Entropy Regularization to regularize the training process, which is to simply add a negative entropy term to the canonical cross-entropy loss without any additional parameters and modification of a model. We theoretically give the convergence probability distribution and analyze how the regularization influence the learning process. Experiments on Chinese character recognition, Chinese text line recognition and fine-grained image classification achieve consistent improvement, proving that the regularization is beneficial to generalization and robustness of a recognition model.
BibTeX
@inproceedings{cheng2020maximum,
  author = {Changxu Cheng and Wuheng Xu and Xiang Bai and Bin Feng and Wenyu Liu},
  title = {{Maximum Entropy Regularization and Chinese Text Recognition}},
  booktitle = {DAS},
  year = {2020},
  doi = {10.1007/978-3-030-57058-3\_1},
}
Figure