Abstract
Fine-Grained Entity Typing (FGET) involves assigning specific semantic types to entity mentions within a textual context. Traditional approaches often rely on static label representations or fixed hierarchies, failing to capture the dynamic interaction between specific instances and their corresponding labels. This paper proposes a novel two-phase heterogeneous graph network that incorporates relational inductive biases to dynamically enhance label representations. By modeling instance-to-label and label-to-label dependencies, the framework bridges the gap between contextual features and abstract categories. Experimental results on OntoNotes, FIGER, and BBN benchmarks demonstrate that the proposed method achieves state-of-the-art performance, highlighting the importance of instance-aware label encoding.
Introduction
Fine-Grained Entity Typing (FGET) is a critical task in natural language processing that requires identifying specific types for entity mentions—for example, classifying "Lily" as an actor rather than just a person. A significant challenge in FGET is the representation of labels. Most existing models treat labels as static vectors or utilize fixed hierarchical structures. However, these methods often overlook the explicit interaction between instances and labels. For instance, contextual cues like "leading role" and "movie" should directly influence the representation of the actor label for a specific mention. This paper addresses this limitation by introducing a dynamic link between instances and labels through relational inductive biases.
Proposed Method
The core of the proposed approach is a two-phase heterogeneous graph network designed to update label representations dynamically for every batch:
- Phase I: Instance-to-Label (Instance-Awareness): A sub-graph is constructed connecting mentions in context to their potential labels. A Graph Attention Network (GAT) is employed to propagate features from instances to label nodes, making the labels "aware" of specific contextual features present in the current data.
- Phase II: Label-to-Label (Dependency-Awareness): To maintain consistency and respect known relationships, a second sub-graph utilizes a Graph Convolutional Network (GCN) to capture dependencies based on label hierarchies and statistical co-occurrence. This "smooths" representations, particularly benefiting low-resource labels.
- Pseudo-Label Generator: Since ground truth labels are unavailable during inference, a pseudo-label generator predicts initial labels to establish the edges for the dynamic graph construction.
Experiments and Results
The model was evaluated on three major FGET benchmarks: OntoNotes (89 labels), FIGER (113 labels), and BBN (47 labels). The proposed GAT-GCN method consistently outperformed previous state-of-the-art models across all datasets, achieving significant gains in Macro-F1 and strict accuracy (ACC).
Key Insights
- Dynamic Labeling: Allowing label representations to shift based on batch instances creates more flexible and accurate decision boundaries compared to static embeddings.
- Dominance of Instance Bias: Ablation studies revealed that Phase I (instance-to-label) contributes more significantly to performance than Phase II, suggesting that contextual signals are the strongest drivers for refining labels.
- Rare Label Support: Information propagation between related labels helps rare categories "borrow" representational strength from common neighbors (e.g., male learning from person), mitigating overfitting on small sample sizes.
Conclusion and Future Work
The proposed relational inductive bias constraint successfully organizes the feature space, as evidenced by superior F1 scores and qualitative t-SNE visualizations showing tighter label clusters. While highly effective, the model's performance is somewhat dependent on the quality of the initial pseudo-label generator. Future work may explore extending this dynamic relational framework to other multi-label classification tasks beyond hierarchical entity typing.
Summary updated