Knowledge Graphs (KGs) like Freebase and DBpedia organize human knowledge into structured triplets $(h, r, t)$. To leverage this data in machine learning, Knowledge Representation (KR) models embed entities and relations into continuous vector spaces. While translation-based models like TransE have seen success, they struggle with the fundamental difference between relational links and intrinsic attributes. This paper introduces KR-EAR, a framework that models relations and attributes separately to improve embedding quality and prediction accuracy.
The "Many-to-One" Challenge in Knowledge Graphs
Traditional KR models typically follow the translation principle: $h + r \approx t$. This geometric assumption works well for one-to-one relations but fails significantly for many-to-one scenarios. Attributes, such as "Gender" or "Nationality," are prime examples of this failure.
As shown in the table above, attributes like "gender" have an extremely high number of head entities per tail ($E_h$), meaning millions of distinct entities point to a single attribute value (e.g., "Male"). Forcing all these diverse entity vectors to satisfy the same translation $h + \text{gender} \approx \text{Male}$ causes their embeddings to collapse toward a single point, destroying the unique structural information of the entities.
The KR-EAR Framework
The authors propose KR-EAR (Knowledge Representation with Entities, Attributes, and Relations) to address this by decoupling the knowledge graph into two distinct layers.
Instead of a unified translation model, KR-EAR employs two specialized encoders:
- Relational Triple Encoder (RTE): Handles structural relations between entities (e.g., FounderOf). It utilizes standard translation models (TransE, TransR) to capture the topology of the KG.
- Attributional Triple Encoder (ATE): Treats attribute prediction as a classification task. By projecting entity embeddings into an attribute-specific space and using a softmax layer, the model avoids the geometric constraints that lead to vector crowding.
Furthermore, KR-EAR models Attribute Correlations. It recognizes that certain attributes are likely to co-occur (e.g., "Profession: Actor" and "Award: Oscar"), allowing the model to infer missing properties through cross-attribute synergy.
Experimental Evaluation
To evaluate KR-EAR, the researchers developed FB24k, a refined version of Freebase where relations and attributes are manually categorized. The dataset contains 23,634 entities and over 400,000 triplets.
The framework was tested against baseline models (TransE, TransH, TransR) on three key tasks:
- Entity Prediction: KR-EAR(TransR) achieved a filtered Mean Rank of 118, a significant improvement over the 200 achieved by the standard TransR.
- Relation Prediction: By leveraging attribute constraints (e.g., a "Parent" must be a "Person"), the model reached 90.4% Hits@1 accuracy.
- Attribute Prediction: This task saw the most dramatic improvement. KR-EAR achieved 70.8% Hits@1, compared to just 55.9% for TransE, proving that classification is a superior objective for modeling entity properties.
Conclusion
KR-EAR demonstrates that not all edges in a knowledge graph are created equal. By distinguishing between the "who" (relations) and the "what" (attributes), the model preserves the distinctiveness of entity embeddings while achieving state-of-the-art performance in fact prediction. Future work aims to automate the categorization of relations and attributes to scale the framework to even larger, noisier datasets.
Summary updated