Abstract
The models developed to date for knowledge base embedding are all based on the assumption that the relations contained in knowledge bases are binary. For the training and testing of these embedding models, multi-fold (or n-ary) relational data are converted to triples (e.g., in FB15K dataset) and interpreted as instances of binary relations. This paper presents a canonical representation of knowledge bases containing multi-fold relations. We show that the existing embedding models on the popular FB15K datasets correspond to a sub-optimal modelling framework, resulting in a loss of structural information. We advocate a novel modelling framework, which models multi-fold relations directly using this canonical representation. Using this framework, the existing TransH model is generalized to a new model, m-TransH. We demonstrate experimentally that m-TransH outperforms TransH by a large margin, thereby establishing a new state of the art.
Introduction: The Binary Assumption
Knowledge Base (KB) embedding is a critical technique for representing entities and relations as continuous vectors, enabling tasks like link prediction and automated question answering. Historically, most models—including benchmarks like TransE and TransH—have relied on a "binary assumption." This assumption posits that every fact in a KB can be reduced to a triple consisting of two entities and a single relation (e.g., Beijing is the capital of China).
However, real-world knowledge is frequently non-binary. A single event or fact often involves multiple entities playing distinct roles. For instance, the fact that "Benedict Cumberbatch played Alan Turing in the movie The Imitation Game" involves an actor, a character, and a film. Forcing such n-ary relations into binary triples is a common but problematic practice that leads to significant information loss.
The Information Loss of Triple Decomposition
To handle n-ary data, researchers often use "Star-to-Clique" (S2C) conversion. This process decomposes an n-ary relation into a set of binary triples representing all possible pairs of participating entities. The authors argue that this approach is fundamentally flawed.
As shown in the figure above, the S2C conversion transforms a star-like structure into a clique. This process is mathematically irreversible; once converted, the original joint n-ary relation cannot be perfectly reconstructed. This suggests that popular datasets like FB15K, which are derived from such conversions, are inherently degraded and contain "noise" that hinders effective learning. Over one-third of entities in Freebase participate in these non-binary relations, making the "binary trap" a significant bottleneck for KB embedding performance.
A Direct Modeling Framework: m-TransH
Instead of decomposing relations, the authors propose a direct modeling framework using a canonical "instance representation." In this view, a multi-fold relation instance is a function mapping specific roles to entities.
Building on this, the authors introduce m-TransH, a generalization of the TransH model designed for n-ary relations. While the original TransH calculates distances between two projected entity vectors, m-TransH evaluates a cost function based on the joint interaction of all entities involved:
- Hyperplane Projection: Entities are projected onto a hyperplane specific to the relation.
- Weighted Contribution: A weight vector ($a_r$) determines the importance of each role within the relation.
- Translation Alignment: The model aims for the weighted sum of these projected entities to align with a relation-specific translation vector ($b_r$).
By modeling the relation directly, m-TransH captures the structural context of the entire fact at once and avoids the combinatorial explosion of triples required by decomposition methods.
Results and the JF17K Dataset
To test their framework, the authors introduced JF17K, a new dataset derived from Freebase that preserves original multi-fold structures. It contains 17,629 entities and 181 multi-fold relation types.
The experimental results were striking. On the JF17K dataset, m-TransH achieved a Hit@10 of over 80%, a massive leap from the ~50% achieved by standard TransH using the triple-based approach. Furthermore, m-TransH significantly improved the Mean Rank, indicating higher precision in its predictions.
The authors also explored adding "Fact IDs"—treating specific instances as entities themselves. While this improved Hit@10, it created a bottleneck in low-dimensional spaces because the model had to embed tens of thousands of unique instances, exhausting the available degrees of freedom.
Conclusion
This research demonstrates that the binary assumption in KB embedding is a significant limitation. By moving from a decomposition framework to a direct modeling framework, m-TransH achieves state-of-the-art results while maintaining computational efficiency. This work opens the door for generalizing other "Trans-series" models to handle the rich, multi-fold complexity of real-world knowledge bases.
Summary updated