Multilingual Event Extraction from Historical Newspaper Adverts

Nadav Borenstein, Natalia da Silva Perez, Isabelle Augenstein

Abstract

This research, originally presented at ACL 2023, explores the automation of detail extraction from 18th-century newspaper advertisements concerning enslaved people seeking freedom. By framing event extraction as an extractive Question Answering (QA) task, the authors demonstrate that modern NLP models can achieve high performance on archaic, multilingual texts despite data scarcity and OCR noise.

Introduction

Historians increasingly rely on digitized archives, but manually "coding" these documents for specific details—such as names, physical descriptions, or rewards—is prohibitively labor-intensive. While Natural Language Processing (NLP) offers a potential solution, historical texts present unique challenges: extreme data scarcity, significant language shift from modern standards, and pervasive Optical Character Recognition (OCR) errors.

The authors focus on "freedom-seeking events," specifically advertisements placed by enslavers to recapture people who had escaped. Extracting these details is crucial for historians to analyze the linguistic processes of racialization and reconstruct the lived experiences of enslaved individuals.

Annotated 18th-century advertisement for a freedom-seeking individual, illustrating the complexity of historical text and entity types.

Methodology: Event Extraction as QA

The study's primary innovation is the formulation of event extraction as an extractive QA task. Rather than traditional sequence labeling (tagging every word), the researchers convert event attributes into natural language questions. For instance, the attribute "Clothing" is queried as "What did the person wear?" The model then identifies the specific text span that answers the question.

A diagram illustrating the conversion of historical advertisement text and attributes into a Question Answering format.

This approach offers several advantages:

  • Versatility: A single model can handle diverse attribute types simultaneously.
  • Verbatim Extraction: The model returns exact spans from the source text, maintaining historical integrity and building trust with domain experts.
  • Transfer Learning: Models pre-trained on modern QA datasets (like SQuAD) provide a robust starting point for historical domains.

Datasets and Multilingual Support

The researchers utilized a combination of existing and new resources across three languages:

  • English: ~800 annotated ads from the Runaway Slaves in Britain project (1700–1780).
  • French: A new evaluation set of 41 ads (Marronage).
  • Dutch: A challenging evaluation set of 44 ads from De Curaçaosche Courant, characterized by high OCR noise.
  • Unlabeled Data: Over 20,000 ads used for semi-supervised "tri-training," where multiple models iteratively teach one another.

Key Results and Insights

The experimental results yielded several significant findings:

  1. High Accuracy: English models achieved an F1 score of approximately 87. Even in zero-shot scenarios, the QA-framed models outperformed standard baselines.
  2. Modern Models for Old Text: Surprisingly, models trained on modern text (e.g., RoBERTa) often outperformed those trained on broad historical corpora. The 18th-century advertising language proved closer to modern English than the legal or literary texts typically used for historical pre-training.
  3. Cross-Lingual Performance: Performance was lower for French (61 F1) and Dutch (52 F1), with the latter heavily impacted by OCR quality. However, machine translation from English training data proved to be a surprisingly strong baseline for these low-resource languages.

Conclusion

This work demonstrates that framing event extraction as QA is a powerful method for historical research. While OCR noise remains a significant bottleneck—particularly for non-English sources—the ability of large language models to generalize to unseen attributes and archaic language opens new doors for large-scale historical analysis. Future research will focus on expanding this methodology to more complex event types and improving robustness against digitized text errors.

Summary updated

Abstract

NLP methods can aid historians in analyzing textual materials in greater volumes than manually feasible. Developing such methods poses substantial challenges though. First, acquiring large, annotated historical datasets is difficult, as only domain experts can reliably label them. Second, most available off-the-shelf NLP models are trained on modern language texts, rendering them significantly less effective when applied to historical corpora. This is particularly problematic for less well studied tasks, and for languages other than English. This paper addresses these challenges while focusing on the under-explored task of event extraction from a novel domain of historical texts. We introduce a new multilingual dataset in English, French, and Dutch composed of newspaper ads from the early modern colonial period reporting on enslaved people who liberated themselves from enslavement. We find that: 1) even with scarce annotated data, it is possible to achieve surprisingly good results by formulating the problem as an extractive QA task and leveraging existing datasets and models for modern languages; and 2) cross-lingual low-resource learning for historical languages is highly challenging, and machine translation of the historical datasets to the considered target languages is, in practice, often the best-performing solution.
BibTeX
@inproceedings{borenstein2023multilingual,
  author = {Nadav Borenstein and Natalia da Silva Perez and Isabelle Augenstein},
  title = {{Multilingual Event Extraction from Historical Newspaper Adverts}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2023},
  doi = {10.18653/v1/2023.acl-long.574},
}
Figure