A Multi-Scale Heterogeneous Graph Attention Network for Nested Named Entity Recognition with Syntactic and Dependency Tree Structures

Yifan Zhao, Lin Zhang, Yangshuyi XuElectronics 2026 · DOI

TL;DR

This paper introduces MHGAT, a novel multi-scale heterogeneous graph attention network designed to tackle the complexities of nested Named Entity Recognition (NER). By integrating syntactic and dependency tree structures into a unified graph, MHGAT effectively captures both local and global dependencies, leading to significant improvements in identifying nested entities, especially long and low-frequency ones.

What Problem Does This Paper Address?

Named Entity Recognition (NER) is a fundamental task in NLP, aiming to identify and categorize entities (like people, organizations, locations) in text. However, real-world text often contains nested entities, where one entity is contained within another, or entities have overlapping boundaries. Traditional NER methods, often framed as sequence labeling, struggle with these complex structures, leading to issues like boundary conflicts, difficulty modeling long-range dependencies, and inadequate representation of deep semantic relationships. This paper addresses the challenge of accurately recognizing these nested entities by proposing a new model that leverages rich structural information from linguistic trees.

Methods

The core of the proposed MHGAT model lies in its innovative approach to integrating structural information and capturing multi-scale dependencies:

  • Syntax and Dependency Tree Integration (SDTI): The model first constructs a unified heterogeneous graph by combining information from both syntactic (phrase structure) and dependency (word-to-word relationships) trees. This provides a rich structural foundation.
  • Multi-Scale Heterogeneous Graph Attention (MHGAT): Instead of simply stacking more layers, MHGAT generates 1-hop, 2-hop, and 3-hop subgraphs. This allows the model to capture information at different receptive field sizes, crucial for handling entities of varying depths and lengths.
  • Multi-Scale Subgraph Attention (MSSA): The model employs attention mechanisms across these multi-scale subgraphs, adaptively fusing information from local (shallow entities) and global (deep entities) contexts. This is a key differentiator from standard GATs that operate on a single graph scale.
  • Depth-Aware Attention Modulation: An attention mechanism is introduced that is modulated by the hierarchical depth of entities, giving more weight to deeper nested structures.
  • Semantic-Structure Residual Fusion: A module combines pre-trained semantic representations (from SpanBERT) with the structural representations learned by MHGAT, ensuring robust and stable feature learning.
  • Predictive Output Layer: A multi-stage process of span filtering, type classification, and hierarchical relation annotation is used to produce the final nested entity output.

Datasets

The authors evaluated MHGAT on several standard datasets:

  • ACE2004, ACE2005, GENIA: These are the primary benchmark datasets used for evaluating the model's performance on nested NER. They cover diverse domains (general and biomedical) and exhibit varying degrees of entity nesting.
  • KBP2017, GermEval2014: These datasets were used to validate the model's generalization capabilities across different domains and languages (English and German).

Key Insights and Ideas

  • The power of dual-tree integration: Combining syntactic and dependency trees into a single graph provides a more comprehensive structural understanding than using either alone, crucial for capturing complex entity relationships.
  • Multi-scale is key for nested entities: Recognizing that entities at different depths require different contextual information, the multi-scale subgraph approach allows the model to adaptively focus on local cues for shallow entities and global dependencies for deep ones. This is more effective than simply increasing GAT depth.
  • Depth matters: Explicitly modeling the hierarchical depth of entities in the attention mechanism helps the model prioritize information from more deeply nested structures, which are often harder to identify.
  • Semantic and structural fusion is vital: Merging pre-trained semantic embeddings with learned structural features leads to more stable and accurate representations, preventing semantic drift and improving overall performance.
  • Careful hyperparameter tuning is critical: Ablation studies revealed optimal settings for feature dimension (d=128), GAT layers (3), and attention heads, highlighting the sensitivity of the model to these configurations.

Key Results

MHGAT demonstrated strong performance across multiple datasets:

  • ACE2004: Achieved an F1 score of 88.9, outperforming most baselines.
  • ACE2005: Achieved an F1 score of 88.3, nearing the performance of top models and showing particular strength in recall for identifying multi-level and nested entities.
  • GENIA: Achieved an F1 score of 82.2, showing robust performance in the challenging biomedical domain, though slightly behind state-of-the-art models in this specific domain.
  • KBP2017 & GermEval2014: Showed strong generalization capabilities, achieving the best overall results on KBP2017 (87.4 F1) and strong performance on GermEval2014 (76.3 F1), demonstrating effectiveness across domains and languages.
  • The model showed particular advantages in identifying long entities and low-frequency entities.

Limitations and Future Work

  • GENIA performance: While strong, MHGAT's performance on the GENIA dataset (biomedical domain) was marginally inferior to some recent models. The authors suggest that further refinement for domain adaptability and more detailed modeling of graph structures could improve efficacy in such domains.
  • Future work: The authors suggest exploring domain adaptability, more nuanced structural designs, and joint optimization of candidate quality and hierarchical consistency as avenues for future research.

Summary updated

Abstract

Nested Named Entity Recognition (nested NER) frequently encounters challenges like boundary conflicts, complications in modeling long-distance dependencies, and inadequate representation of deep nested semantics resulting from overlapping spans and hierarchical inclusion relationships of entities. This research presents a multi-scale heterogeneous graph attention network to facilitate end-to-end recognition of nested entities through the collaborative modeling of structure and semantics. The model initially presents the structural integration mechanism, which consolidates the hierarchical restrictions of the syntactic tree and the inter-word relationships of the dependency tree within a singular heterogeneous graph space. It subsequently generates 1/2/3-hop multi-scale subgraphs and employs multi-scale subgraph attention to adaptively integrate information from various structural receptive fields, harmonizing the local cues of shallow entities with the global dependencies of deep entities. The experimental findings on the ACE2004, ACE2005, and GENIA benchmark datasets indicate that the proposed method surpasses several robust baselines regarding overall performance and nested entity recognition, particularly exhibiting notable advantages in identifying long entities and low-frequency entities. We further evaluate MHGAT on KBP2017 and GermEval2014 to validate generalization across datasets and languages.
BibTeX
@inproceedings{zhao2026multi,
  author = {Yifan Zhao and Lin Zhang and Yangshuyi Xu},
  title = {{A Multi-Scale Heterogeneous Graph Attention Network for Nested Named Entity Recognition with Syntactic and Dependency Tree Structures}},
  booktitle = {Electronics},
  year = {2026},
  doi = {10.3390/electronics15061183},
}
Figure