Abstract
This paper introduces pFedCR, a personalized federated learning framework specifically designed for Chinese Optical Character Recognition (OCR). Chinese OCR faces significant challenges due to the vast number of character categories and the "long-tail" distribution of data across different organizations. Furthermore, privacy concerns prevent direct data sharing of sensitive documents. pFedCR addresses these issues through a two-stage approach: Global Model Training with server-side virtual data balancing and Local Personalization using attention-based fine-tuning. Experimental results demonstrate a ~20% performance improvement over standalone training and a 6-8% lead over state-of-the-art federated methods, effectively handling "few-shot" characters while maintaining data privacy.
Introduction
Chinese text recognition is a "long-tail" challenge. Unlike Latin scripts with a few dozen characters, Chinese involves thousands of categories. In real-world scenarios—such as medical records or financial invoices—a single organization might only see a small fraction of these characters frequently, leading to poor performance on "few-shot" or rare characters.
To improve performance, organizations typically need to pool their data. However, text data is inherently sensitive, often containing addresses, names, and ID numbers. Standard Federated Learning (FL) aims to solve the privacy issue by sharing model weights instead of raw data. However, standard FL often struggles with non-IID data (where different clients have vastly different writing styles) and the massive class imbalance inherent in Chinese text.
Methodology: The pFedCR Framework
The proposed pFedCR (Personalized Federated Chinese Recognition) framework operates in two distinct stages to balance global knowledge with local specialization.
Stage 1: Global Model Training (GMT)
The framework utilizes a CRNN (Convolutional Recurrent Neural Network) backbone augmented with ECA (Efficient Channel Attention) layers. To combat the long-tail problem, the server uses a public "Text Renderer" to generate a balanced virtual dataset. Before aggregating client weights, the server fine-tunes the global model on this synthetic data to ensure it retains knowledge of rare characters. Additionally, during local training, clients freeze the last embedding layer to prevent local data imbalances from warping the global classification head.
Stage 2: Local Personalization (LPS)
Once a robust global model is established, it is personalized for each client's specific style (e.g., a hospital's specific form layout) through two steps:
- Fine-tuning & Averaging: The global model is fine-tuned on local data, and the weights are averaged with the global weights to retain general knowledge.
- Attention Fine-tuning: The main CRNN parameters are frozen, and only the ECA attention layers are fine-tuned using a regularization term. This allows the model to adapt to local stylistic features without losing the robust feature extraction capabilities learned during the global stage.
Experimental Evaluation
The study evaluates pFedCR on three industrial-grade datasets: Document (scanned text), Receipt (mobile-captured photos), and HikCSIG (six types of invoices). The researchers simulated real-world institutional silos using both 3-client and 9-client settings.
Key Results and Insights
- Performance Gains: pFedCR improved local model performance by approximately 20% compared to standalone local training and outperformed other popular personalized FL methods (like FedAvg-ft, FedBN, and Ditto) by 6% to 8%.
- Solving the Few-Shot Problem: For characters a client had never seen in their local training set, pFedCR achieved accuracies as high as 61.6%, whereas local models scored 0%.
- Synthetic Data as a Privacy Anchor: By using synthetic data on the server, the model remains balanced without ever seeing private client data. The authors also argue that OCR models are naturally more privacy-resilient than image classifiers due to the sequential and semantic nature of text.
- Attention for Style: The paper demonstrates that attention layers are the most effective place to inject personalization, allowing the model to adapt to local fonts and backgrounds efficiently.
Conclusion and Future Work
pFedCR provides a strong empirical defense of privacy while significantly improving Chinese OCR performance. Future work may involve implementing formal Differential Privacy (DP) to further harden the framework against potential attacks, as well as adapting the system to open-vocabulary scenarios or extremely rare ancient scripts.
Summary updated