Digitizing historical records is a massive undertaking for governments worldwide. In Uruguay, the transition from paper-based birth certificates to digital databases is a critical step in ensuring legal identity for all citizens. However, transcribing these handwritten documents manually is slow and error-prone.
A recent study investigates how deep learning can automate this process, specifically focusing on a fundamental question in data labeling: Should models be trained on "clean," standardized data (Normalized) or verbatim, character-for-character transcriptions (Diplomatic)?
The Annotation Dilemma: Normalized vs. Diplomatic
The core challenge of Automatic Information Extraction (AIE) from handwritten documents lies in the trade-off between two annotation strategies:
- Normalized Annotation: Labels follow a strict, computer-friendly format. For example, converting a handwritten "31" into the text "treinta y uno" to match existing database entries. This forces the model to perform recognition and translation simultaneously.
- Diplomatic Annotation: Labels reflect exactly what is on the page, including specific casing, accents, and abbreviations (e.g., transcribing "Maria J." exactly as written rather than expanding it to "Maria Juana").
Methodology: The Document Attention Network (DAN)
The researchers employed the Document Attention Network (DAN), an end-to-end architecture that bypasses the need for manual segmentation of document fields. DAN utilizes a fully convolutional network as an encoder and an auto-regressive transformer decoder, allowing the model to "attend" to different parts of the document image to generate tokens for specific fields.
To overcome the limitations of a small dataset, the model was pre-trained on the RIMES 2009 dataset (French handwritten letters) and then fine-tuned on Uruguayan birth certificates. This transfer learning approach allowed the model to achieve high accuracy with only ~160 training images.
Experimental Setup
The study utilized a dataset of 201 scanned Uruguayan birth certificates from various years (2008–2016), encompassing more than 15 different writing styles. The extraction task targeted several key fields, including document numbers, names of the enrollees and parents, and dates of enrollment.
Results and Discussion
The results, measured by Character Error Rate (CER), revealed a significant performance gap between the two strategies. The Diplomatic DAN performed better overall, with a global normalized CER of 2.04%, compared to 4.32% for the Normalized DAN.
The "Translation" Burden and the Name Gap
Normalized annotation proved difficult for the model because it adds a "translation" layer. If a date is written as a numeral but the label is a word, the model must learn to read the visual input and then map it to a different textual representation.
This was most evident in unpredictable fields like names. For the "2nd parent's name" field, the Diplomatic DAN achieved a CER of 5.70%, while the Normalized DAN struggled at 15.81%. Because names do not follow a standard dictionary, the diplomatic approach—relying purely on visual evidence—is far more robust.
Conversely, for highly predictable fields like "year of enrollment" or "jurisdiction," the Normalized DAN was effective, often achieving near 0% error. This suggests that the choice of annotation should depend on the field's complexity.
Conclusion
This study demonstrates that while normalized labels are useful for predictable, standardized fields, diplomatic (verbatim) annotation is essential for complex fields like names. The researchers propose a hybrid annotation strategy for future production systems: using normalized labels for dates and locations to simplify post-processing, while maintaining diplomatic labels for names to ensure maximum recognition accuracy. This approach balances the need for database compatibility with the inherent strengths of end-to-end deep learning models.
Summary updated