Abstract
Constituency parsing for historical languages is often hindered by extreme data scarcity and a significant lexical gap between ancient and modern variants. This paper presents a delexicalized approach to parsing Middle High German (MHG) by leveraging the structural similarities between Modern German (MG) and its ancestor. By training a span-based parser on part-of-speech (POS) sequences and morphological features rather than raw tokens, the proposed method achieves a 67.3% F1-score. This represents a 28.6 percentage point improvement over standard zero-shot transfer baselines, demonstrating that syntactic structure is a more robust bridge for cross-lingual transfer than lexical overlap in diachronic contexts.
Introduction
Constituency parsing—the hierarchical decomposition of sentences into grammatical components like noun and verb phrases—is a fundamental task in Natural Language Processing (NLP). While modern languages benefit from massive annotated treebanks, ancient languages like Middle High German (spoken c. 1050–1350) suffer from a "formidable" lack of data. The primary challenge is twofold: first, the scarcity of annotated resources (fewer than 100 trees exist for MHG); and second, the failure of standard zero-shot transfer methods.
State-of-the-art models like mBERT or gBERT rely heavily on subword overlaps. However, because spelling, phonology, and vocabulary have shifted dramatically over the last millennium, the lexical gap between Modern German and Middle High German is too wide for these models to bridge effectively. This study explores an alternative: ignoring the words entirely and focusing on the underlying "structural DNA" of the language.
Methodology: The Delexicalized Approach
The core hypothesis of this work is that while vocabulary changes rapidly, the syntactic structure of a language family remains relatively stable. The authors propose a Delexicalized Approach consisting of several key stages:
- Delexicalization: The training process strips away all raw words. Instead of learning from a sentence like "Die Bänke sind klein" (The benches are small), the model is trained on sequences of POS tags and morphological features (e.g.,
ART.Nom.Pl.Fem,NN.Nom.Pl.Fem,VAFIN). - Source Training: The Berkeley Neural Parser (Benepar), a span-based parser, is trained on a delexicalized version of the Tiger Treebank, which contains approximately 50,000 annotated trees of Modern German.
- Target Tagging and Mapping: To process MHG text, an RNNTagger (trained on the 2.3-million-token ReM corpus) generates POS tags. Because the MHG tagger uses the HiTS tagset while the Modern German treebank uses STTS, the authors developed a mapping dictionary to normalize the inputs for the parser.
- Cross-Lingual Transfer: The parser, having mastered the "grammar of tags" from Modern German, is applied directly to the mapped tags of Middle High German without any further fine-tuning on the target language.
Experimental Setup and Datasets
The study utilized three primary datasets:
- Tiger Treebank (MG): The source for training the delexicalized parser.
- German Diachronic Treebank (DDB): A small but essential evaluation set containing 96 manually annotated MHG trees.
- Reference Corpus for Middle High German (ReM): Used to train the high-quality POS tagger required for the target language.
Results and Key Insights
The delexicalized parser (Dexparser) significantly outperformed all baselines. It achieved an F1-score of 67.3% on the MHG test set, a staggering 28.6 point increase over the best zero-shot transformer baseline (Tetra-gBERT). Additionally, the model achieved a Complete Match (CM) rate of 12.5%, meaning it perfectly reconstructed the entire tree structure for one out of every eight sentences.
The Importance of Morphology
A critical finding of the ablation study was the role of morphological information. When the model was trained on POS tags alone, the F1-score dropped by 11.5%. In a morphologically rich language like German, features such as case, gender, and number are vital for disambiguating complex syntax, especially when lexical cues are absent.
The Failure of PLMs
The study highlighted that Pretrained Language Models (PLMs) like gBERT and mBERT performed poorly on this task. This suggests that for ancient languages, "clever" feature engineering and structural focus can still outperform the "brute force" scaling of large-scale transformers, which are often blinded by the lexical shifts of a millennium.
Discussion and Limitations
While highly effective, the delexicalized approach is not without limitations. Its performance is strictly capped by the accuracy of the POS tagger used on the target language; errors in tagging propagate directly to the parser. Qualitative analysis also revealed that the model struggles with specific syntactic complexities, such as separated verb prefixes and ambiguous adverbs in long sentences.
Furthermore, while the delexicalized model is slightly less accurate on Modern German than a standard word-based model (83.1% vs 87.2%), its robustness across time makes it the superior choice for diachronic transfer. Future research may explore "semi-lexicalized" models that incorporate shared linguistic roots to further refine parsing accuracy in low-resource historical contexts.
Summary updated