Abstract
Relation Extraction (RE) typically relies on deterministic knowledge graphs, which often fail to capture the inherent ambiguity and hierarchical nature of real-world entities. This paper introduces the MIUK (Multi-view Inference with Uncertain Knowledge) framework, which integrates local linguistic context with global probabilistic taxonomies. By utilizing uncertain knowledge graphs like ProBase, MIUK assigns confidence scores to entity concepts, providing a richer prior for relationship identification. Experimental results on DocRED and ACE2005 demonstrate that MIUK achieves state-of-the-art performance by effectively balancing local and global information.
Introduction
Relation Extraction (RE) is fundamental to building structured knowledge from unstructured text. Traditional approaches identify semantic relationships (e.g., leader_of) between entities (e.g., Bill de Blasio and New York City) using deterministic Knowledge Graphs (KGs). In these systems, a relationship is binary: it either exists or it does not.
However, entities are often multifaceted. "New York" can be a "City," a "State," or a "Tourist Destination" depending on the context. Deterministic KGs struggle with this ambiguity. MIUK addresses this by incorporating Uncertain KGs, which provide confidence scores for relationships. This probabilistic approach allows the model to resolve relations more accurately, especially when the local text is vague or limited.
The MIUK Framework
The core of MIUK is a multi-view inference strategy that synthesizes information across three distinct levels of representation:
- Mention-View (Local): This view uses BERT to encode specific entity mentions within the document, capturing immediate linguistic context and syntactic dependencies.
- Entity-View (Global/Semantic): To provide broader semantic grounding, this view incorporates textual descriptions of entities retrieved from external sources like Wikipedia.
- Concept-View (Global/Taxonomic): This view leverages ProBase, an uncertain KG, to retrieve the top-$K$ concepts associated with an entity. These concepts are weighted by their KG confidence scores to create a "global interactive vector."
System Architecture
The MIUK architecture facilitates complex interactions between these views to arrive at a final relation prediction.
Cross-view Links and Gating
Information flows between views via specialized mechanisms: M2E (Mention-to-Entity), E2C (Entity-to-Concept), and C2E (Concept-to-Entity). A key innovation is the Gating Mechanism, a learnable component that dynamically weights the importance of local contextual information ($u_l$) versus global external knowledge ($u_g$). This allows the model to rely on the text when it is clear and lean on the KG when the text is ambiguous.
Mixed Attention
For document-level extraction, MIUK employs a Mixed Attention mechanism. It combines information-based weights with empirical weights—determined by whether an entity explicitly appears in a sentence—to aggregate multiple sentence representations into a single document-level vector.
Experimental Evaluation
The model was evaluated on two major benchmarks: DocRED (document-level) and ACE2005 (sentence-level). Additionally, the authors introduced ProBase_Desp, a new corpus of Wikipedia descriptions for over 15 million entities to support the Entity-View.
Key Results
- DocRED: MIUK achieved an F1 score of 59.99%, outperforming strong baselines such as LSR (59.05%) and GEDA (55.74%).
- ACE2005: The model reached an F1 of 75.7%, setting a new state-of-the-art and significantly exceeding BERT-only baselines (72.0%).
- Ablation Study: Removing the uncertainty component (treating all KG concepts as equally likely) resulted in a ~1.0 drop in F1, confirming that probabilistic weighting is essential.
Conclusion
MIUK demonstrates that uncertainty in knowledge graphs is a valuable feature rather than a limitation. By balancing local mentions with global taxonomic concepts through a gated multi-view framework, the model achieves superior robustness in relation extraction. Future work will explore more complex graph-based interactions between mentions and concepts to further refine the hierarchical modeling of uncertain knowledge.
Summary updated