Multilingual Event Extraction from Historical Newspaper Adverts

Nadav Borenstein, Natalia da Silva Perez, Isabelle Augenstein

Introduction

Historians managing large-scale archives face a significant "big data" challenge: extracting structured information from millions of digitized documents. Manually identifying details such as names, physical descriptions, or destinations from historical advertisements is labor-intensive and requires specialized domain knowledge. While Natural Language Processing (NLP) offers potential solutions, historical texts introduce unique hurdles, including data scarcity, significant domain shifts from modern language models, and a lack of tools for non-English languages.

This research focuses on "freedom-seeking events"—advertisements placed by enslavers in the 18th century to recapture escaped individuals. Extracting these details is crucial for understanding historical processes of racialization and the lived experiences of enslaved people.

Methodology: Event Extraction as Extractive QA

The primary innovation of this work is reframing event extraction as an extractive question-answering (QA) task. Traditional event extraction involves identifying "triggers" and filling complex schemas. In contrast, this approach treats a historical advertisement as the "context" and the desired attributes (e.g., "Total reward") as "questions" (e.g., "How much reward is offered?").

Conceptual diagram of the extractive QA approach for historical event extraction.

This formulation provides two key advantages:

  1. Verbatim Extraction: Models extract text directly from the source, maintaining historical fidelity and avoiding the hallucinations common in generative models.
  2. Transfer Learning: It enables the use of powerful transformer models (like RoBERTa) pre-trained on massive modern QA datasets (e.g., SQuAD), which can then apply their underlying logic to historical contexts.

The researchers evaluated several architectures, including multilingual models (XLM-RoBERTa) and specialized historical language models such as MacBERTh and BERT-hist-news.

Datasets and Multilingual Strategy

The study utilizes datasets across English (en), French (fr), and Dutch (nl):

  • Runaway Slaves in Britain: An annotated English dataset (~800 ads, 1700–1780), which was machine-translated to create synthetic training data for other languages.
  • Marronage & Delpher: Large unannotated collections used for semi-supervised learning.
  • New Multilingual Evaluation Dataset: A core contribution consisting of 41 French and 44 Dutch ads manually annotated by professional historians to serve as a "gold standard."

Key Results and Analysis

The experiments revealed that framing the problem as QA is a powerful strategy for low-resource historical data. In English, a RoBERTa-large model fine-tuned on SQuAD achieved an F1 score of 56.42 in a zero-shot setting, outperforming specialized baselines like OneIE.

F1 scores across different languages and models as a function of training dataset size.

Key findings include:

  • Few-Shot Efficiency: With as few as 16 labeled examples, English models reached F1 scores in the high 70s.
  • Translation vs. Cross-Lingual Transfer: Machine-translating training data into the target language and using a monolingual model often proved more effective than using complex cross-lingual models like XLM-R.
  • Temporal Alignment: Surprisingly, models trained on 19th-century news sometimes outperformed those trained on much older texts (like MacBERTh), suggesting that training data distribution relative to the target century is critical.
  • Multilingual Performance: While zero-shot performance was lower for French (F1 ~44) and Dutch (F1 ~40), semi-supervised techniques like tri-training significantly boosted results.

Conclusion and Future Directions

While the extractive QA approach shows great promise for historical event extraction, limitations remain. The current study focuses on a single event type and Western European languages. Future work should explore generalizability to other document types (e.g., court records), improve robustness against OCR noise, and investigate "question-tuning" to optimize prompt effectiveness.

Summary updated

BibTeX
@inproceedings{borenstein2023multilingual,
  author = {Nadav Borenstein and Natalia da Silva Perez and Isabelle Augenstein},
  title = {{Multilingual Event Extraction from Historical Newspaper Adverts}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2023},
  doi = {10.18653/v1/2023.acl-long.574},
}
Figure