Introduction
Knowledge Graph Completion (KGC) is a critical task in artificial intelligence, aimed at predicting missing relational links between entities in large-scale datasets. Tensor Decomposition (TD) has emerged as a powerful approach for KGC, yet it faces a persistent challenge: the trade-off between model complexity and expressive power. Dense core tensors, while highly expressive, are prone to overfitting and parameter explosion. Conversely, sparse or diagonal core tensors are efficient but often fail to capture complex semantic interactions. This paper presents SPAC (Sparse Partitioning and Adaptive Core tensor pruning), a novel model that bridges this gap through a hybrid core structure and dynamic pruning.
Methodology
The SPAC framework is built upon three innovative components that allow it to remain lean while maintaining high representational capacity.
1. Sparse Partitioning
SPAC moves away from the binary choice of dense versus sparse core tensors. Instead, it utilizes a hybrid partitioning strategy. The model features Main Cores, which are dense blocks designed to capture precise, fine-grained semantic associations, and Auxiliary Cores, which are sparse units used for general, coarse-grained information. This dual-structure ensures that parameters are concentrated where they are most needed.
2. Interaction Enhancement
A common limitation in block-diagonal models is the lack of interaction between different embedding dimension groups. SPAC introduces an Intermediate Output Gating mechanism to solve this. By applying a linear gating function, the model forces interaction between isolated partition groups and adds beneficial non-linearity, significantly boosting the model's expressive power.
3. Adaptive Core Tensor Pruning
To ensure long-term efficiency, SPAC employs a dynamic pruning strategy during training. The model evaluates dimensions based on their importance (calculated via the Frobenius norm) and diversity (variance across facts). Dimensions that contribute minimally to the model's predictive power are pruned and replaced with simpler auxiliary units. This allows the model to evolve and "slim down" its architecture to fit the specific characteristics of the dataset.
Experimental Evaluation
The authors evaluated SPAC on three standard KGC benchmarks: FB15k-237 (general world knowledge), WN18RR (lexical relations), and YAGO3-10 (large-scale relational data). These datasets provide a rigorous testing ground for both accuracy and scalability.
Results and Discussion
SPAC achieves state-of-the-art (SOTA) accuracy across all tested datasets. Notably, on FB15k-237, it achieved a Hits@1 score of 0.280, surpassing previous benchmarks set by models like MEIM and Tucker.
Beyond raw accuracy, the efficiency gains are substantial. SPAC is approximately 2x faster than the standard Tucker model while utilizing only half the parameters. Furthermore, ablation studies indicate that SPAC's gating and pruning mechanisms are highly versatile; they can be integrated into other existing models like DistMult or ComplEx to improve their performance and efficiency.
Conclusion
SPAC offers a sophisticated solution to the efficiency-expressivity dilemma in Knowledge Graph Completion. By combining hybrid sparse partitioning with adaptive pruning and enhanced interaction gating, it provides a model that is both leaner and smarter than traditional dense alternatives. Future research will focus on scaling SPAC to industrial-sized graphs and exploring its integration with pretrained Large Language Models to further enhance semantic understanding.
Summary updated