Introduction
Ancient manuscripts are vital for historical and socio-political research, but digitizing them presents significant technical challenges. While Google’s Tibetan OCR has processed millions of pages for the Buddhist Digital Resource Center (BDRC), the resulting text is often riddled with errors. These errors stem from visual ambiguities in handwritten characters and physical decay—such as faded ink and woodblock artifacts—that create "noisy" digital outputs unsuitable for scholarly search or citation.
Methods: Integrating OCR Confidence Scores
Researchers from Harvard and MIT have developed "Cleansing Jewel," a Transformer-based model designed to post-process and correct these OCR errors by treating the problem as a specialized Grammatical Error Correction (GEC) task.
The core innovation of this model is the Confidence Score mechanism. Standard OCR engines output a confidence score (0.0 to 1.0) for every recognized character, but this metadata is typically discarded. "Cleansing Jewel" instead integrates these scores directly into the model's embedding layer.
As illustrated in the architecture, the model concatenates standard word/character embeddings with "confidence score embeddings." These embeddings are manually initialized based on the OCR score to facilitate faster learning. The researchers found that maintaining high granularity (using 101 buckets to preserve float precision) performed significantly better than simplified binary or 5-bucket schemes, indicating that nuanced signals from the OCR engine are critical for correction.
Datasets and Data Alignment
The study utilized a dataset provided by the BDRC, consisting of raw JSON outputs from Google’s Tibetan OCR and corresponding human-corrected ground truth versions.
The noisy syllables were paired with clean data at the sentence level to create a parallel corpus. For tokenization, the team employed Byte-Pair Encoding (BPE) with a vocabulary size of 300–500, which provided the optimal balance between character-level precision and semantic context.
Key Results and Insights
The "Cleansing Jewel" model significantly outperformed standard sequence-to-sequence architectures. By leveraging the OCR engine's internal uncertainty, the model learns to focus its corrective "attention" on tokens with low confidence scores.
- CER Reduction: The initial OCR output had a 25% Character Error Rate (CER). The Cleansing Jewel model reduced this to 12.26%.
- Architecture Comparison: The model outperformed standard Transformers (17.68% CER), as well as LSTM-to-LSTM (47.53% CER) and GRU-to-GRU (48.31% CER) baselines.
- Efficiency: The Confidence Score mechanism improved loss by 23.26% and CER by 30.66% compared to a vanilla Transformer.
Visualization of self-attention heatmaps revealed that the model primarily attends to 2–3 neighboring tokens. While this is effective for fixing local spelling mistakes and removing non-textual noise (like random symbols), it limits the model's ability to resolve errors requiring broader semantic context.
Limitations and Future Work
Despite the improvements, the model struggles with semantic ambiguity—words that are spelled correctly but are contextually wrong. Future research directions include scaling the BPE vocabulary to 2000–3000 tokens, integrating Tibetan-specific pre-trained models like BERT for global context, and combining NLP post-processing with computer vision denoising techniques.
Summary updated