Federated Knowledge Graph Completion via Latent Embedding Sharing and Tensor Factorization

Maolin Wang, Dun Zeng, Zenglin Xu, Ruocheng Guo, Xiangyu Zhao

Abstract

Knowledge Graph Completion (KGC) is essential for addressing the inherent incompleteness of real-world relational data. However, data silos and privacy regulations often prevent the centralized aggregation of Knowledge Graphs (KGs). This paper introduces FLEST (Federated Latent Embedding Sharing and Tensor factorization), a novel framework for privacy-preserving KGC. FLEST decomposes embedding matrices into shared latent "dictionaries" and private "loadings," allowing collaborative learning without exposing sensitive entity or relation identities. Experimental results on standard benchmarks demonstrate that FLEST outperforms existing federated methods in both privacy and accuracy, maintaining robustness even under high data dispersion.

Introduction

Knowledge Graphs (KGs) are critical structures for representing real-world entities and their relationships. Despite their utility, KGs are almost always incomplete. While KGC—the task of predicting missing links—is well-studied, practical implementation is often hindered by data silos. Organizations such as hospitals or financial institutions are restricted by privacy regulations from sharing their local KGs, even though their data is often complementary.

Existing Federated Learning (FL) approaches for KGs suffer from significant drawbacks. For instance, FedE requires a central server to track entity IDs, risking information leakage. FedR struggles with low relation overlap between clients, and FKGE relies on computationally expensive and unstable adversarial training (GANs). FLEST aims to overcome these limitations by providing a computationally efficient, robust, and privacy-preserving alternative.

Methodology

FLEST treats the KG as a third-order adjacency tensor (Subject × Relation × Object) and employs a probabilistic version of Tucker Decomposition for graph completion. The core innovation lies in its approach to embedding matrix management.

Matrix Partitioning and Latent Sharing

Instead of sharing raw embeddings, FLEST partitions the embedding matrix into two distinct components:

  1. Dictionary Matrix ($E_{dic}$): A shared set of latent features representing the "basis" of global knowledge.
  2. Loading Matrix ($E_{loading}$): A client-specific matrix that determines how to combine latent features for local entities.

During training, only the Dictionary Matrix and fusion weights are transmitted to the central server for aggregation via FedAvg. Because the Loading Matrix remains local, the server cannot map latent features back to specific entities, providing a strong layer of privacy through indirection.

Optimization and Efficiency

To ensure the dictionary captures meaningful patterns, the authors apply orthogonality constraints to keep features distinct and sparsity constraints to ensure entities utilize only relevant features. To manage the high parameter count of Tucker decomposition, FLEST utilizes CP decomposition to reduce the core tensor's complexity, making it feasible for clients with limited hardware.

Tensor factorization of the Knowledge Graph Tensor using decomposed components.

Experimental Setup

The model was evaluated on two benchmark datasets: FB15k-237 (a subset of Freebase) and WN18RR (a subset of WordNet). Federated environments were simulated by partitioning these datasets across 5 to 20 clients to test performance under varying degrees of data dispersion.

Results and Discussion

FLEST demonstrated competitive performance in standalone settings, matching the accuracy of state-of-the-art non-federated models like TuckER. In multi-client federated tests, FLEST consistently outperformed existing methods. For example, on the FB15k-237 dataset with 5 clients, FLEST achieved a Mean Reciprocal Rank (MRR) of 0.127, significantly higher than local DistMult (0.067) or local TuckER (0.121).

Key insights from the results include:

  • Robustness to Disjoint Data: The latent dictionary allows clients to benefit from global feature patterns even when their specific entities and relations are unique.
  • Stability: Moving away from adversarial training toward a decomposition-based approach resulted in a more stable and mathematically grounded training process.

Conclusion

FLEST provides a robust framework for federated knowledge graph completion that prioritizes both privacy and performance. While performance naturally degrades as data dispersion increases across more clients, FLEST remains more resilient than its predecessors. Future work will explore personalized embeddings for heterogeneous data structures and conduct active security testing to further quantify the privacy protections offered by the dictionary-sharing mechanism.

Summary updated

BibTeX
@inproceedings{wang2023federated,
  author = {Maolin Wang and Dun Zeng and Zenglin Xu and Ruocheng Guo and Xiangyu Zhao},
  title = {{Federated Knowledge Graph Completion via Latent Embedding Sharing and Tensor Factorization}},
  booktitle = {ICDM},
  year = {2023},
  url = {https://openreview.net/forum?id=hDhVDztPlFp},
}
Figure