Abstract
Named Entity Recognition (NER) is a fundamental task in Natural Language Processing, yet many languages like Danish suffer from a lack of large-scale annotated datasets. This paper investigates the effectiveness of neural cross-lingual transfer from English to Danish, combined with minimal target-language annotation. By employing a BiLSTM-CRF architecture and unsupervised embedding alignment, the study demonstrates that state-of-the-art performance can be achieved with as few as 10,000 annotated tokens.
Introduction
While deep learning has significantly advanced NER in high-resource languages like English, these models typically require massive amounts of labeled data. For Danish, publicly available human-labeled datasets are scarce, hindering the development of robust NLP tools. This research addresses the "data desert" by exploring two primary questions: the feasibility of "borrowing" knowledge from English and the minimum amount of Danish-specific data required to achieve high accuracy.
Methods and Transfer Strategies
The core model utilizes a BiLSTM-CRF architecture. To facilitate cross-lingual transfer, the author employs MUSE for unsupervised alignment, mapping Danish Polyglot embeddings into the English vector space without a parallel dictionary. Several learning strategies were evaluated:
- Zero-shot Transfer: Training on English and testing directly on Danish.
- Joint Training: Simultaneous training on English and Danish data (referred to as neural transfer).
- Fine-tuning: Pre-training on English followed by tuning on Danish samples.
As shown in the results, joint training (neural transfer) generally outperforms fine-tuning in low-resource settings. The inclusion of pre-trained Polyglot embeddings (+Poly) also provides a significant performance boost over "plain" neural models trained on limited data.
Data and Experimental Setup
The study uses the English CoNLL 2003 dataset as the source and the Danish Universal Dependencies treebank (Danish-DDT) as the target. A key contribution is the creation of a new NER layer for the Danish-DDT development and test sets. Training was conducted on two small subsets: TINY (approx. 5,000 tokens) and SMALL (approx. 10,000 tokens).
Key Results
The proposed neural approach establishes a new state-of-the-art for Danish NER. The best-performing configuration (Medium English source + SMALL Danish data) achieved an F1 score of 70.8 on the development set, significantly outperforming existing rule-based and HMM-based systems.
The results indicate that the neural transfer model is particularly effective for categories like PER (Person) and LOC (Location). Even in a zero-shot setting, the model achieved an F1 of ~61%, highlighting the structural similarities in named entities across these languages.
Conclusion
This work demonstrates that massive datasets are not a prerequisite for high-quality NER in lower-resource languages. By leveraging cross-lingual transfer and annotating a modest amount of data (10,000 tokens), researchers can build competitive models that surpass traditional baselines. Future research may explore domain robustness and the impact of transferring from more linguistically similar source languages like Swedish or Norwegian.
Summary updated