Abstract
Named Entity Recognition (NER) has greatly advanced by the introduction of deep neural architectures. However, the success of these methods depends on large amounts of training data. The scarcity of publicly-available human-labeled datasets has resulted in limited evaluation of existing NER systems, as is the case for Danish. This paper studies the effectiveness of cross-lingual transfer for Danish, evaluates its complementarity to limited gold data, and sheds light on performance of Danish NER.
Introduction
Named Entity Recognition (NER)—the task of identifying names, locations, and organizations in text—is a cornerstone of modern Natural Language Understanding. While deep learning has pushed NER performance to new heights, these models are notoriously data-hungry. For languages like Danish, there is a significant gap: while the language is well-resourced in general, it lacks the massive, publicly available, human-annotated NER datasets required to train state-of-the-art neural models. This paper investigates how to build effective Danish NER systems when only a minimal amount of gold-standard data is available, primarily by leveraging neural cross-lingual transfer from English.
Methodology
The research employs a BiLSTM-CRF architecture, which served as the standard for neural sequence labeling. To bridge the gap between English (source) and Danish (target), the author utilizes two core strategies:
- Cross-Lingual Mapping: Using Polyglot embeddings aligned into a shared space via unsupervised "Procrustes rotation" (using the MUSE library). This allows a model trained on English to interpret Danish words based on their relative geometric positions in the embedding space.
- Transfer Learning Setups: The study compares several approaches: Zero-shot (training on English, testing on Danish), In-language (training only on small Danish samples), Joint Training (simultaneous training on English and Danish), and Fine-tuning (pre-training on English followed by Danish refinement).
Datasets
The source data is the classic CoNLL 2003 English NER dataset. For the target language, the author introduces a new gold-standard evaluation set by annotating the Danish section of the Universal Dependencies (UD) treebank. To simulate low-resource scenarios, the author defines TINY (5k tokens) and SMALL (10k tokens) training portions. A notable finding is the "density challenge": while English CoNLL data is entity-dense (~80% of sentences contain entities), the Danish UD data is more lexically rich but less dense (~35-39% of sentences contain entities), representing a more realistic linguistic environment.
Key Results and Analysis
The experiments reveal that neural transfer methods significantly outperform traditional approaches and limited in-language training.
As shown in the table above, Joint Training (neural transfer) proved more effective than standard fine-tuning in low-resource settings. The best Danish NER results were achieved using joint training with the "SMALL" dataset, reaching an F1 score of 70.82. Furthermore, the zero-shot baseline (F1 ~58-61%) proved remarkably formidable, easily outperforming a neural model trained only on the "TINY" Danish dataset (36.17%).
The proposed neural transfer model also outperformed existing Danish systems like DKIE and the built-in Polyglot tagger. However, a per-entity breakdown shows that while the model excels at identifying Persons (PER) and Locations (LOC), it struggles significantly with the Miscellaneous (MISC) category, which remains a common pain point in NER tasks.
Conclusion
This study demonstrates that neural cross-lingual transfer is a highly effective strategy for Danish NER in the face of limited annotated data. Joint training on source and target languages provides superior results to sequential fine-tuning, and aligned embeddings enable strong zero-shot performance. Future work will likely involve exploring transformer-based models like mBERT and transferring from linguistically closer languages such as Swedish or Norwegian to further improve accuracy, particularly for challenging entity categories like MISC.
Summary updated