Abstract
This research, originally presented at ACL 2023, explores the automation of detail extraction from 18th-century newspaper advertisements concerning enslaved people seeking freedom. By framing event extraction as an extractive Question Answering (QA) task, the authors demonstrate that modern NLP models can achieve high performance on archaic, multilingual texts despite data scarcity and OCR noise.
Introduction
Historians increasingly rely on digitized archives, but manually "coding" these documents for specific details—such as names, physical descriptions, or rewards—is prohibitively labor-intensive. While Natural Language Processing (NLP) offers a potential solution, historical texts present unique challenges: extreme data scarcity, significant language shift from modern standards, and pervasive Optical Character Recognition (OCR) errors.
The authors focus on "freedom-seeking events," specifically advertisements placed by enslavers to recapture people who had escaped. Extracting these details is crucial for historians to analyze the linguistic processes of racialization and reconstruct the lived experiences of enslaved individuals.
Methodology: Event Extraction as QA
The study's primary innovation is the formulation of event extraction as an extractive QA task. Rather than traditional sequence labeling (tagging every word), the researchers convert event attributes into natural language questions. For instance, the attribute "Clothing" is queried as "What did the person wear?" The model then identifies the specific text span that answers the question.
This approach offers several advantages:
- Versatility: A single model can handle diverse attribute types simultaneously.
- Verbatim Extraction: The model returns exact spans from the source text, maintaining historical integrity and building trust with domain experts.
- Transfer Learning: Models pre-trained on modern QA datasets (like SQuAD) provide a robust starting point for historical domains.
Datasets and Multilingual Support
The researchers utilized a combination of existing and new resources across three languages:
- English: ~800 annotated ads from the Runaway Slaves in Britain project (1700–1780).
- French: A new evaluation set of 41 ads (Marronage).
- Dutch: A challenging evaluation set of 44 ads from De Curaçaosche Courant, characterized by high OCR noise.
- Unlabeled Data: Over 20,000 ads used for semi-supervised "tri-training," where multiple models iteratively teach one another.
Key Results and Insights
The experimental results yielded several significant findings:
- High Accuracy: English models achieved an F1 score of approximately 87. Even in zero-shot scenarios, the QA-framed models outperformed standard baselines.
- Modern Models for Old Text: Surprisingly, models trained on modern text (e.g., RoBERTa) often outperformed those trained on broad historical corpora. The 18th-century advertising language proved closer to modern English than the legal or literary texts typically used for historical pre-training.
- Cross-Lingual Performance: Performance was lower for French (61 F1) and Dutch (52 F1), with the latter heavily impacted by OCR quality. However, machine translation from English training data proved to be a surprisingly strong baseline for these low-resource languages.
Conclusion
This work demonstrates that framing event extraction as QA is a powerful method for historical research. While OCR noise remains a significant bottleneck—particularly for non-English sources—the ability of large language models to generalize to unseen attributes and archaic language opens new doors for large-scale historical analysis. Future research will focus on expanding this methodology to more complex event types and improving robustness against digitized text errors.
Summary updated