Abstract
Fine-grained Entity Typing (FET) is essential for advanced natural language understanding but faces significant challenges due to complex label hierarchies and structural noise in distantly supervised datasets. This paper introduces FBTree, a framework that jointly models hierarchical label dependencies and structural label noise. By employing a "Forward Tree" (FTree) to predict true entity types and a "Backward Tree" (BTree) to model label corruption, FBTree achieves state-of-the-art performance on major benchmarks like Wiki and BBN.
Introduction
Fine-grained Entity Typing (FET) involves assigning specific, hierarchical labels to entity mentions in text—for instance, identifying "Demi Lovato" as PERSON.ARTIST rather than just PERSON. This specificity is vital for downstream applications like question answering and information extraction.
However, FET is hindered by two primary issues:
- Complex Hierarchies: Labels are not independent; they exist in tree structures where capturing nuances between siblings (e.g.,
ACTORvs.DIRECTOR) is difficult. - Structural Label Noise: Large-scale datasets often rely on Distant Supervision (DS), which can introduce noise. Existing models often treat this noise as random, failing to recognize that errors typically occur between "nearby" labels in the hierarchy.
Methodology: The FBTree Framework
The proposed FBTree framework addresses these challenges by decoupling the prediction of true types from the modeling of observed noise.
Forward Tree (FTree): The Informative Walk
The FTree module models the "true" type distribution $P(Y|x)$. It treats type selection as a random walk starting from the root. At each node, a scoring function—comparing entity features to type embeddings—determines whether to stop or proceed to a child node. To ensure mathematical consistency, "stop nodes" are added to the hierarchy, effectively normalizing it so every valid type terminates at a leaf.
Backward Tree (BTree): The Erroneous Walk
The BTree is the core innovation, modeling how a "true" label $Y$ might be perturbed into an "observed" noisy label $Z$. It simulates a walker that can move backward (up to a parent), across (to a sibling), or remain stationary. This process generates a constrained confusion matrix $P(Z|Y)$. Unlike dense matrices that overfit, BTree’s matrix is strictly governed by the tree's structure, significantly reducing the parameter space.
The model is trained by maximizing the likelihood that the true distribution, when passed through the noise matrix, matches the noisy labels in the training data.
Experiments and Results
The researchers evaluated FBTree on three standard benchmarks: Wiki (FIGER), OntoNotes, and BBN. These datasets vary in size and noise levels (ranging from 24% to 35% noisy mentions).
Performance and Robustness
FBTree consistently outperformed previous state-of-the-art models. On the Wiki dataset, it achieved 70.7% accuracy, and on BBN, it reached 77.3%. A key finding was the model's stability as entity types became more specific.
As shown in the figure above, while most models experience a sharp performance drop at deeper levels of the hierarchy, FBTree maintains significantly higher F1 scores at Level 2 and Level 3. Ablation studies further confirmed that removing the BTree component leads to a consistent performance decline, highlighting the necessity of explicit noise modeling.
Discussion and Conclusion
The success of FBTree stems from two key insights:
- Noise is Structural: Label errors in hierarchical systems are rarely random; they are usually "nearby" in the taxonomy. Modeling noise as a walk on the tree allows the model to effectively "undo" these mistakes.
- Decoupling Truth from Error: By separating the informative walk from the erroneous walk, the model learns cleaner entity representations even from unreliable training data.
While FBTree is highly effective, it remains dependent on the quality of the pre-defined hierarchy. Future work may explore extending this "Forward-Backward" approach to other hierarchical classification domains, such as image categorization or functional genomics.
Summary updated