Domain-Adapted Dependency Parsing for Cross-Domain Named Entity Recognition

Chenxiao Dou, Xianghui Sun, Yaoshu Wang, Yunjie Ji, Baochang Ma, Xiangang LiAAAI 2023 · OpenReview

Abstract

Named Entity Recognition (NER) often struggles with domain shifts due to vocabulary changes. This paper introduces DADP (Domain-Adapted Dependency Parsing), a framework that leverages the cross-domain stability of syntactic structures. By incorporating dependency parsing as an auxiliary task in a multi-task learning setup and employing unsupervised domain adaptation via Maximum Mean Discrepancy (MMD), DADP learns to identify entities through universal structural patterns. Experiments demonstrate that DADP significantly outperforms existing methods on challenging datasets like WNUT17 and shows remarkable robustness to noise.

Introduction

While state-of-the-art NER models perform exceptionally well within a single domain, their performance typically degrades when applied to new domains (e.g., from news to social media). A promising solution lies in utilizing Dependency Parsing (DP), as entity boundaries often align with syntactic sub-trees. However, most NER datasets lack dependency annotations. Previous attempts to use pre-predicted labels often fail due to error propagation in out-of-domain text.

The DADP Framework

DADP addresses these challenges through a unified Multi-Task Learning (MTL) architecture:

  1. Shared Representation: A pretrained BERT model followed by a BiLSTM layer provides a robust contextual foundation for both tasks.
  2. Auxiliary DP Task: A biaffine classifier is trained on high-resource source data (e.g., OntoNotes) to predict word-dependency relations.
  3. NER Task: The NER module identifies entity spans by integrating structural features learned from the DP module.
  4. Domain Adaptation: To bridge the gap between source and target domains, the model uses Maximum Mean Discrepancy (MMD) to align the distributions of syntactic features, ensuring they remain useful in the target domain without requiring target-domain DP labels.

Syntactic Consistency and Boundary Alignment

The core intuition behind DADP is that while word meanings vary across domains, grammatical structures remain relatively consistent. Furthermore, named entities frequently correspond to specific branches in a dependency tree.

Figure 1: Dependency structures for 'The Cape of Good Hope' in different contexts (Art vs. Location).

As shown in Figure 1, the internal dependency structure of "The Cape of Good Hope" remains identical whether it functions as a "Location" or a work of "Art." By learning these structural regularities, DADP can accurately identify entity boundaries even when encountering unfamiliar vocabulary in a new domain.

Experimental Results

The researchers evaluated DADP across several benchmarks, including CoNLL03 (News), WNUT17 (Social Media), and NCBI (Biomedical).

  • State-of-the-Art Performance: On the noisy WNUT17 dataset, DADP achieved an F1 score of 45.93, a substantial improvement over the BiLSTM-CRF baseline (39.18).
  • Robustness: When random noise was introduced into the CoNLL03 dataset, DADP's performance remained stable, whereas traditional models saw significant declines. This suggests that the model's reliance on syntax allows it to "see through" surface-level noise.
  • Data Efficiency: DADP showed superior performance in few-shot scenarios, maintaining high F1 scores even when only 10% of target-domain training data was available.

Conclusion

DADP demonstrates that treating dependency parsing as a learned auxiliary objective, rather than a static input feature, provides a powerful mechanism for cross-domain NER. By aligning syntactic features across domains, the model captures universal structural patterns that facilitate robust entity recognition in low-resource and noisy environments. Future work will explore more advanced domain adaptation techniques to further narrow the distribution gap between disparate text sources.

Summary updated

Abstract

In recent years, many researchers have leveraged structural information from dependency trees to improve Named Entity Recognition (NER). Most of their methods take dependency-tree labels as input features for NER model training. However, such dependency information is not inherently provided in most NER corpora, making the methods with low usability in practice. To effectively exploit the potential of word-dependency knowledge, motivated by the success of Multi-Task Learning on cross-domain NER, we investigate a novel NER learning method incorporating cross-domain Dependency Parsing (DP) as its auxiliary learning task. Then, considering the high consistency of word-dependency relations across domains, we present an unsupervised domain-adapted method to transfer word-dependency knowledge from high-resource domains to low-resource ones. With the help of cross-domain DP to bridge different domains, both useful cross-domain and cross-task knowledge can be learned by our model to considerably benefit cross-domain NER. To make better use of the cross-task knowledge between NER and DP, we unify both tasks in a shared network architecture for joint learning, using Maximum Mean Discrepancy(MMD). Finally, through extensive experiments, we show our proposed method can not only effectively take advantage of word-dependency knowledge, but also significantly outperform other Multi-Task Learning methods on cross-domain NER. Our code is open-source and available at https://github.com/xianghuisun/DADP.
BibTeX
@inproceedings{dou2023domain,
  author = {Chenxiao Dou and Xianghui Sun and Yaoshu Wang and Yunjie Ji and Baochang Ma and Xiangang Li},
  title = {{Domain-Adapted Dependency Parsing for Cross-Domain Named Entity Recognition}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2023},
  url = {https://openreview.net/forum?id=ypE5Zgt8DF},
}
Figure