HDiff: Confidence-Guided Denoising Diffusion for Robust Hyper-relational Link Prediction

Xiangfeng Luo, Ruoxin Zheng, Jianqiang Huang, Hang YuEMNLP 2025

TL;DR

This paper introduces HDiff, a novel diffusion-based framework for robustly predicting missing links in complex, noisy knowledge graphs called Hyper-relational Knowledge Graphs (HKGs). By leveraging a novel denoising diffusion process guided by both local confidence and global context, HDiff significantly outperforms existing methods, especially in the presence of noise.

What Problem Does This Paper Address?

Traditional knowledge graphs (KGs) represent facts as simple triples (subject, relation, object). However, many real-world scenarios require richer representations, leading to Hyper-relational Knowledge Graphs (HKGs). HKGs extend triples with "qualifiers" (key-value pairs) that provide crucial context, allowing for more nuanced and complex factual statements. While this richness is powerful, it also introduces new types of noise: Intra-fact Inconsistency (contradictions within a single hyper-fact) and Cross-fact Association Noise (spurious links between hyper-facts due to misleading qualifiers). Existing methods for Hyper-relational Knowledge Graph Completion (HKGC), which aim to predict missing elements in these hyper-facts, often struggle with this noise, as they are typically designed for cleaner data and lack mechanisms to explicitly handle these specific HKG noise patterns.

Methods

HDiff tackles HKG noise with a two-pronged approach:

  1. Consistency-Enhanced Global Encoder (CGE): This module generates robust initial embeddings for entities, relations, and hyper-facts. It uses a Graph Attention Network (GAT) for its flexibility in handling varying qualifier importance. Crucially, it employs contrastive learning (Barlow Twins) to enforce semantic consistency between the full hyper-fact representation and its core triple representation. This directly combats Intra-fact Inconsistency. A TransE loss is also used to anchor embeddings in fundamental relational semantics.
  2. Context-Guided Denoiser (CGD): This is the core of HDiff, employing a conditional denoising diffusion probabilistic model (DDPM). It takes the robust embeddings from CGE and iteratively reconstructs clean hyper-facts. The novelty lies in its dual conditioning:
  3. Local Confidence Condition (clc): A learned confidence score identifies potentially inconsistent components within a hyper-fact, guiding the denoiser to focus on these areas.
  4. Global Context Condition (che): The robust hyper-fact embedding from CGE provides global structural context, ensuring the reconstructed fact is coherent with the overall graph.

This diffusion process, guided by both local uncertainty and global structure, allows HDiff to effectively denoise and reconstruct hyper-facts, even when they are corrupted by noise.

Datasets

The paper evaluates HDiff on three standard hyper-relational knowledge graph benchmarks:

  • JF17K
  • WikiPeople
  • WD50K

Key Insights and Ideas

  • HKG Noise is Unique: The paper clearly identifies and characterizes two specific noise types (Intra-fact Inconsistency and Cross-fact Association Noise) that are distinct from traditional KG noise and pose significant challenges for HKGC.
  • Diffusion for Denoising: Applying diffusion models, typically used for generative tasks like image synthesis, to the problem of denoising noisy knowledge graph facts is a novel and effective approach.
  • Dual Conditioning is Key: The success of the CGD module hinges on its ability to leverage both local confidence signals (identifying problematic parts) and global structural context (ensuring overall coherence). This synergistic approach is crucial for robust reconstruction.
  • Contrastive Learning for Consistency: Using Barlow Twins for contrastive learning within the CGE is a clever way to enforce semantic coherence within individual hyper-facts, directly addressing Intra-fact Inconsistency.

Key Results

  • State-of-the-Art Performance: HDiff consistently achieves state-of-the-art (SOTA) performance on entity prediction across all three benchmark datasets (JF17K, WikiPeople, WD50K), outperforming strong baselines like GRAN and HAHE. For example, on JF17K (subject/object), HDiff's MRR of 0.637 surpasses HAHE's 0.623.
  • Superior Robustness: HDiff demonstrates significantly enhanced robustness under increasing synthetic noise levels compared to existing methods. While other models degrade substantially, HDiff's performance drops more gracefully.
  • Ablation Studies Validate Components: Removing key components like the Barlow Twins loss (w/o BT) or the global context condition (w/o HE) leads to significant performance degradation, especially under noise, highlighting the critical role of each module.
  • Diffusion Module's Impact: An ablation study shows that the diffusion process (CGD) significantly mitigates performance degradation in noisy settings, reducing the drop from 22.8% to 11.8% compared to a model without diffusion.

Limitations and Future Work

  • Computational Cost: The diffusion process and Transformer architectures inherent in HDiff lead to higher computational costs for training and inference compared to simpler models. This could be a challenge for resource-constrained environments or extremely large graphs.
  • Hyperparameter Sensitivity: Like many neural models, HDiff's performance can be sensitive to hyperparameter choices, requiring careful tuning.
  • Limited Noise Range: The robustness analysis was primarily conducted on synthetic noise levels between 2% and 20%. The model's behavior under significantly higher or qualitatively different noise distributions remains unexplored.
  • Future Work: The authors suggest exploring efficiency improvements, broader hyper-relational reasoning tasks, and integrating LLMs into HKGC for tasks like context generation or noise identification, complementing task-specific models.

Summary updated

BibTeX
@inproceedings{luo2025hdiff,
  author = {Xiangfeng Luo and Ruoxin Zheng and Jianqiang Huang and Hang Yu},
  title = {{HDiff: Confidence-Guided Denoising Diffusion for Robust Hyper-relational Link Prediction}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2025},
}
Figure