Enhancing Label Representations with Relational Inductive Bias Constraint for Fine-Grained Entity Typing

Jinqing Li, Xiaojun Chen, Dakui Wang, Yuwei Li

Abstract

Fine-Grained Entity Typing (FGET) involves assigning specific semantic types to entity mentions within a textual context. Traditional approaches often rely on static label representations or fixed hierarchies, failing to capture the dynamic interaction between specific instances and their corresponding labels. This paper proposes a novel two-phase heterogeneous graph network that incorporates relational inductive biases to dynamically enhance label representations. By modeling instance-to-label and label-to-label dependencies, the framework bridges the gap between contextual features and abstract categories. Experimental results on OntoNotes, FIGER, and BBN benchmarks demonstrate that the proposed method achieves state-of-the-art performance, highlighting the importance of instance-aware label encoding.

Introduction

Fine-Grained Entity Typing (FGET) is a critical task in natural language processing that requires identifying specific types for entity mentions—for example, classifying "Lily" as an actor rather than just a person. A significant challenge in FGET is the representation of labels. Most existing models treat labels as static vectors or utilize fixed hierarchical structures. However, these methods often overlook the explicit interaction between instances and labels. For instance, contextual cues like "leading role" and "movie" should directly influence the representation of the actor label for a specific mention. This paper addresses this limitation by introducing a dynamic link between instances and labels through relational inductive biases.

Proposed Method

The core of the proposed approach is a two-phase heterogeneous graph network designed to update label representations dynamically for every batch:

  1. Phase I: Instance-to-Label (Instance-Awareness): A sub-graph is constructed connecting mentions in context to their potential labels. A Graph Attention Network (GAT) is employed to propagate features from instances to label nodes, making the labels "aware" of specific contextual features present in the current data.
  2. Phase II: Label-to-Label (Dependency-Awareness): To maintain consistency and respect known relationships, a second sub-graph utilizes a Graph Convolutional Network (GCN) to capture dependencies based on label hierarchies and statistical co-occurrence. This "smooths" representations, particularly benefiting low-resource labels.
  3. Pseudo-Label Generator: Since ground truth labels are unavailable during inference, a pseudo-label generator predicts initial labels to establish the edges for the dynamic graph construction.
Architecture of the proposed two-phase heterogeneous graph network for FGET, illustrating the Mention Features Extractor, Pseudo-label Generator, and Instance-Aware Label Encoder.

Experiments and Results

The model was evaluated on three major FGET benchmarks: OntoNotes (89 labels), FIGER (113 labels), and BBN (47 labels). The proposed GAT-GCN method consistently outperformed previous state-of-the-art models across all datasets, achieving significant gains in Macro-F1 and strict accuracy (ACC).

Performance comparison of the proposed GAT-GCN method against state-of-the-art models on OntoNotes, FIGER, and BBN datasets.

Key Insights

  • Dynamic Labeling: Allowing label representations to shift based on batch instances creates more flexible and accurate decision boundaries compared to static embeddings.
  • Dominance of Instance Bias: Ablation studies revealed that Phase I (instance-to-label) contributes more significantly to performance than Phase II, suggesting that contextual signals are the strongest drivers for refining labels.
  • Rare Label Support: Information propagation between related labels helps rare categories "borrow" representational strength from common neighbors (e.g., male learning from person), mitigating overfitting on small sample sizes.

Conclusion and Future Work

The proposed relational inductive bias constraint successfully organizes the feature space, as evidenced by superior F1 scores and qualitative t-SNE visualizations showing tighter label clusters. While highly effective, the model's performance is somewhat dependent on the quality of the initial pseudo-label generator. Future work may explore extending this dynamic relational framework to other multi-label classification tasks beyond hierarchical entity typing.

Summary updated

Abstract

Fine-Grained Entity Typing (FGET) is a task that aims at classifying an entity mention into a wide range of entity label types. Recent researches improve the task performance by imposing the label-relational inductive bias based on the hierarchy of labels or label co-occurrence graph. However, they usually overlook explicit interactions between instances and labels which may limit the capability of label representations. Therefore, we propose a novel method based on a two-phase graph network for the FGET task to enhance the label representations, via imposing the relational inductive biases of instance-to-label and label-to-label. In the phase 1, instance features will be introduced into label representations to make the label representations more representative. In the phase 2, interactions of labels will capture dependency relationships among them thus make label representations more smooth. During prediction, we introduce a pseudo-label generator for the construction of the two-phase graph. The input instances differ from batch to batch so that the label representations are dynamic. Experiments on three public datasets verify the effectiveness and stability of our proposed method and achieve state-of-the-art results on their testing sets.
BibTeX
@inproceedings{li2021enhancing,
  author = {Jinqing Li and Xiaojun Chen and Dakui Wang and Yuwei Li},
  title = {{Enhancing Label Representations with Relational Inductive Bias Constraint for Fine-Grained Entity Typing}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2021},
  doi = {10.24963/ijcai.2021/529},
}

Loading related content...

Figure