Abstract
Knowledge graphs (KGs) are widely used to facilitate relation extraction (RE) tasks. While most previous RE methods focus on leveraging deterministic KGs, uncertain KGs, which assign a confidence score for each relation instance, can provide prior probability distributions of relational facts as valuable external knowledge for RE models. This paper proposes to exploit uncertain knowledge to improve relation extraction. Specifically, we introduce ProBase, an uncertain KG that indicates to what extent a target entity belongs to a concept, into our RE architecture. We then design a novel multi-view inference framework to systematically integrate local context and global knowledge across three views: mention-, entity- and concept-view. The experimental results show that our model achieves competitive performances on both sentence- and document-level relation extraction, which verifies the effectiveness of introducing uncertain knowledge and the multi-view inference framework that we design.
1. Introduction
Relation Extraction (RE) aims to identify semantic relationships between entities in a given text. While many existing RE models boost their performance using external Knowledge Graphs (KGs), they typically rely on deterministic KGs (e.g., FreeBase) where a relation either definitively exists or doesn't. However, real-world knowledge is often ambiguous or probabilistic. Uncertain KGs, which assign confidence scores to relations, offer richer, nuanced prior knowledge. This paper introduces MIUK (Multi-view Inference with Uncertain Knowledge), a framework that effectively captures and integrates this probabilistic external knowledge with the local context of a document.
2. Methodology: The MIUK Framework
The MIUK architecture systematically fuses information across three distinct views using BERT as the underlying text encoder:
- Mention-view: Aggregates all mention embeddings of an entity using an attention mechanism guided by external entity descriptions.
- Entity-view: Represents the local contextual information of the entity synthesized from the mention-view.
- Concept-view: Retrieves the top-$K$ related concepts for an entity from ProBase (an uncertain KG). It aggregates these concepts using their prior confidence scores to form a global knowledge representation.
The model uses cross-view links (Mention2Entity, Entity2Concept, Concept2Entity) to pass information between these levels. A gating mechanism dynamically balances the local interactive vector (from the text) and the global interactive vector (from the KG). Finally, a mixed attention mechanism generates the final document representation for multi-label relation classification.
3. Experimental Evaluation
The authors evaluated MIUK on standard benchmarks: DocRED for document-level RE and ACE 2005 for sentence-level RE. They also introduced ProBase_Desp, a corpus containing high-quality Wikipedia text descriptions for over 15 million entities and concepts found in ProBase.
Key Results
- Document-Level (DocRED): MIUK achieved an F1 score of 59.99%, outperforming the previous state-of-the-art (LSR) by nearly 1.0 absolute F1.
- Sentence-Level (ACE 2005): MIUK achieved a Micro-F1 of 75.7%, setting a new state-of-the-art and beating a strong BERT-based baseline by a 3.7% absolute F1 margin.
- Ablation Highlights: Removing the cross-view inference framework caused a severe >2.0% drop in F1. Furthermore, using confidence scores from the uncertain KG was found to be crucial; treating the KG as deterministic noticeably degraded performance.
4. Discussion and Conclusion
The success of MIUK demonstrates that uncertainty is a feature, not a bug. Incorporating rich text descriptions and probabilistic priors allows the model to systematically bridge the gap between raw text mentions and abstract global knowledge.
While effective, the model is sensitive to the amount of external knowledge; adding more than three concepts ($K > 3$) from ProBase introduces noise that can hurt performance. Additionally, the approach relies on high-quality descriptions; without them, the model falls back to generic entity types. Future work will investigate more complex interactions among views and apply this framework to other NLP tasks.
Summary updated