Introduction
Traditional Knowledge Graphs (KGs) typically operate under a "closed-world" assumption, where facts are binary: either true (present in the dataset) or false (absent). However, real-world knowledge is inherently uncertain. Uncertain Knowledge Graphs (UKGs) address this by assigning confidence scores to triplets (head entity, relation, tail entity). For instance, a medical KG might record a drug's efficacy for a symptom with a confidence of 0.8.
Existing UKG embedding models face significant challenges. Most notably, the False-Negative Problem arises when models treat all unseen triplets as having zero confidence. In reality, many missing triplets are plausible but simply absent from the training data. Furthermore, previous architectures like UKGE were often rigid, tied to specific scoring functions, and sometimes relied on non-scalable manual logic rules.
To address these issues, the authors propose PASSLEAF (Pool-bAsed Semi-Supervised LEArning Framework), a novel approach that leverages semi-supervised learning and a "sample pool" to better estimate the confidence of unseen facts.
Methods
The PASSLEAF framework introduces two primary technical innovations: Generalized Score Mapping and the Sample Pool mechanism.
Generalized Score Mapping
PASSLEAF is designed to be model-agnostic. It provides a mathematical bridge to adapt various embedding models—including semantic-based models (e.g., ComplEx, DistMult) and translational-distance models (e.g., RotatE)—into uncertainty-aware versions. By employing a sigmoid-based mapping, the framework ensures that predicted scores are constrained between 0 and 1, representing confidence levels.
The Sample Pool and Semi-Supervised Learning
The core of PASSLEAF is its treatment of unseen triplets. Instead of labeling them as zero-confidence negatives, the model treats them as semi-supervised samples.
- Estimation: The model estimates confidence scores for unseen triplets.
- Sample Pool: These triplets and their estimated scores are stored in a "Sample Pool." This pool acts as a temporal memory, capturing the model's learned knowledge across different training epochs.
- Training: By training on these estimated scores, the model avoids the "false-negative" trap. This process functions as a "temporal ensemble," where the model learns from its own past predictions to stabilize the learning process.
Datasets
The effectiveness of PASSLEAF was evaluated across five datasets:
- PPI5K: High-density protein-protein interaction data.
- NL27K: General knowledge from the NELL database.
- CN15K: Commonsense knowledge from ConceptNet.
- WN18RR & FB15K237: Standard deterministic benchmarks used to verify performance on certain data.
Key Insights
- Unseen ≠ Impossible: In uncertain graphs, the noise from treating unseen triplets as zero-confidence is particularly damaging. Semi-supervised estimation provides "soft" labels that preserve potential knowledge.
- Temporal Ensembling via the Pool: The sample pool prevents overfitting to the model's most recent state. By including samples scored at different training stages, the pool provides a more robust training signal.
- Discovery of Missing Facts: Qualitative analysis shows the model can correctly assign high confidence to facts missing from the training set, such as
(croatia, part_of, europe).
Key Results
PASSLEAF demonstrates significant improvements over the previous state-of-the-art (UKGE):
- Tail Entity Prediction: On CN15K, PASSLEAF reduced the Weighted Mean Rank (WMR) by 50% compared to UKGE.
- Ranking Accuracy: It achieved a 4% improvement in nDCG on the NL27K dataset.
- Ablation Study: Removing the Sample Pool led to a significant performance drop, confirming its role in stabilizing semi-supervised learning.
Limitations and Future Work
While effective, PASSLEAF introduces new hyperparameters, such as pool size and the semi-supervised ramp-up rate, which require careful tuning. Additionally, predicting scores for a large pool of samples increases computational overhead compared to simple negative sampling. Future research could explore automated hyperparameter optimization and more efficient sampling strategies to further enhance the framework.
Summary updated