Abstract
Named Entity Recognition (NER) for historical documents faces significant challenges due to Optical Character Recognition (OCR) errors and a lack of annotated data for medieval languages. This paper presents a novel bootstrapping pipeline that leverages modern abstracts of medieval charters to generate a massive "silver-standard" multilingual corpus of 3.6 million sentences. By fine-tuning transformer-based models on this data, the authors achieve high-accuracy NER for late medieval Czech, Latin, and German, bridging the gap between clean modern summaries and noisy historical texts.
Introduction
Identifying proper names—such as individuals and locations—is a fundamental task in digital humanities. While modern NER systems perform exceptionally well on contemporary news corpora, their performance degrades significantly when applied to historical documents. Two primary obstacles hinder progress: the "OCR Gap," where digitization errors corrupt entity names, and "Data Scarcity," as manual annotation of medieval texts requires specialized paleographic expertise and is prohibitively expensive. This research addresses the "cold start" problem by developing a scalable method to create high-quality training sets for medieval NER without extensive manual labor.
The Bootstrapping Pipeline
The authors propose an iterative pipeline to generate "silver-standard" labels. The process begins with a "Seed" dataset: a small set of modern Czech abstracts of medieval charters manually annotated by historians. Using these entities as queries, an Information Retrieval (IR) system—specifically the "Manatee" system—identifies occurrences of the same entities within 3.6 million sentences of raw OCR text from historical books.
This initial matching creates the Books-Small dataset. Subsequently, the pipeline expands iteratively. An initial model trained on the small set is used to infer entities in the remaining text, creating progressively larger datasets (Books-Medium and Books-Large). The final models are based on XLM-RoBERTa, utilizing a multi-objective approach that combines Masked Language Modeling with Token Classification. To address class imbalance, a weighted cross-entropy loss function was employed to prioritize the correct identification of rare entity tags.
Corpus Development
The hierarchy of corpora developed in this study ranges from manually annotated "gold" data to massive "silver" datasets. These cover a linguistic mix of medieval Czech (43%), Latin (36%), and German (17%).
The statistics in Table 1 highlight the scale of the Books-Huge corpus, which provides the volume of data necessary for deep learning models to learn the linguistic nuances and OCR patterns of the Hussite era.
Experimental Results
The evaluation focused on comparing different model sizes and training configurations. The results demonstrate that "historical-aware" training is essential for high performance.
As shown in Table 2, the best-performing model (Model L) achieved an F-score of 91.61% on the Abstracts-Tiny set and 93.43% on the Books test set. Notably, the smaller Model S performed nearly as well as the larger version, suggesting that model tuning and data quality are more critical than raw parameter count for this specific domain. The use of Weighted Cross-Entropy (WCE) loss was found to be non-negotiable for achieving high recall on rare entity classes.
Key Insights and Discussion
A major finding is the efficacy of "silver" data; models trained on IR-bootstrapped noisy text generalized better than those trained solely on clean, modern data. The "Manatee" IR system was particularly effective due to its ability to handle complex inflections in Czech and Latin. While the model showed high precision (up to 95.6% for Czech), it faced more challenges with German (70.8% precision), likely reflecting the training data distribution.
Limitations and Future Work
Despite the success of the pipeline, some limitations remain. The quality of the 3.6M sentence Books-Huge corpus has not been fully audited, and boundary errors—where the model incorrectly includes surrounding punctuation or titles—persist. Furthermore, the models are currently optimized for the Hussite era (1419–1436); their applicability to other medieval periods requires further investigation. The authors have released the corpus, models, and code to facilitate further research in historical NER.
Summary updated