Introduction
Knowledge Graphs (KGs) such as Freebase and DBpedia represent facts as triples $(h, r, t)$. Traditional representation learning models, like TransE, typically embed entities into a single, static vector space. However, entities are inherently multi-faceted; their relevant characteristics change depending on the context of the relation. For example, "William Shakespeare" functions as an author when associated with a play, but as an award nominee in the context of a ceremony.
Existing methods often ignore hierarchical type information or use it merely as a post-processing filter. This paper introduces TKRL (Type-embodied Knowledge Representation Learning), which explicitly models these hierarchies to produce more nuanced embeddings, especially for sparse, "long-tail" data.
Methodology
TKRL extends the translation-based framework ($h + r \approx t$) by introducing type-specific projection matrices. Instead of a single vector, an entity is projected into a specific "type space" determined by the relation's requirements.
Hierarchical Encoders
To capture the structure of type hierarchies (e.g., Poet $\rightarrow$ Author $\rightarrow$ Person), the authors propose two encoding mechanisms:
- Recursive Hierarchy Encoder (RHE): This treats the hierarchy as a chain of operations, where a sub-type's projection is the product of all its parent type matrices.
- Weighted Hierarchy Encoder (WHE): This uses a weighted sum of type matrices, applying a "proportional-declined" strategy that prioritizes specific sub-types over general categories.
Soft Type Constraints (STC)
During training, TKRL utilizes Soft Type Constraints for negative sampling. By intentionally selecting "wrong" entities that share the same type as the correct answer, the model is forced to learn subtle distinctions between similar entities, enhancing its discriminative power.
Evaluation and Results
The models were evaluated on the standard FB15K benchmark and a new FB15K+ dataset designed to simulate real-world "long-tail" distributions.
As shown in the results table, TKRL variants consistently outperform state-of-the-art baselines. On FB15K, TKRL (WHE+STC) achieved a Hits@10 of 73.4%, a significant improvement over TransR's 67.2%. The model's strength is most evident in long-tail scenarios; for relations appearing 10 times or fewer, TKRL improved Hits@10 by an absolute 10.1% over TransR. Additionally, TKRL achieved a triple classification accuracy of 88.5%.
Conclusion
TKRL demonstrates that treating entity types as dynamic projections rather than static constraints significantly improves the quality of knowledge graph embeddings. By leveraging hierarchical priors, the model becomes more resilient to data sparsity and noise. Future research will explore integrating additional multi-source information, such as text descriptions and images, into this hierarchical framework.
Summary updated