Introduction
Knowledge Base Completion (KBC) is a fundamental task in artificial intelligence, focused on predicting missing facts within a graph of entities and relations. Traditional compositional models, such as DistMult and HolE, represent entities as static "type" embeddings. While effective, these static vectors often fail to capture the context-dependent nature of entities—their "guises"—which vary depending on the specific relation involved. This paper introduces Gradient Graphs, a framework that transforms these static representations into context-sensitive "token" embeddings through a process of semantic optimization.
The Gradient Graph Framework
The authors propose a post-compositional optimization layer inspired by Harmonic Grammar and Optimality Theory. The process begins with a standard compositional model to generate an initial triplet embedding $x$.
The core of the method is the Harmony Function, $H(h, x)$, which evaluates a hidden state $h$ based on two criteria:
- Core Harmony: A measure of internal semantic coherence, asking if the semantic features of the triplet "belong" together based on learned weights.
- Faithfulness: A penalty term, scaled by a hyperparameter $\lambda$, that ensures the optimized state $h$ remains grounded in the original compositional input $x$.
By maximizing this Harmony function, the model derives a "token" embedding $\hat{h}$. Because the function is quadratic, the authors provide a closed-form solution, enabling efficient inference without the need for iterative gradient descent.
Experimental Results
The proposed "Harmonic" versions of existing models, HDistMult and HHolE, were evaluated on the standard FB15K and WN18 benchmarks.
As shown in the results table, the Harmonic variants demonstrate significant improvements across multiple metrics. On the FB15K dataset, HHolE achieved a Hits@10 of 0.901 and a Hits@1 of 0.727, notably outperforming the base HolE model. The improvements were particularly pronounced in the Hits@1 metric, which requires the model to predict the exact correct entity, showing gains of up to 15% over reimplemented baselines. Furthermore, the HHolE model matched or exceeded the performance of several complex ensemble models.
Conclusion
The Gradient Graph approach provides a powerful mechanism for capturing supracompositional nuances in knowledge bases. By shifting from static "type" embeddings to context-aware "token" embeddings, the model achieves state-of-the-art performance while maintaining interpretability. While the closed-form solution involves a matrix inversion that may pose challenges for extremely high-dimensional spaces, the framework offers a robust alternative to simply increasing model depth or complexity.
Summary updated