In the early days of Knowledge Graph Embedding (KGE), models like TransE treated entities and relations as static points in a vector space. The intuition was simple: if a fact exists as a triple $(h, r, t)$, then the vector for the head entity plus the vector for the relation should approximately equal the vector for the tail entity ($h + r \approx t$).
While elegant, this "one-vector-per-entity" approach suffers from a fundamental flaw: it assumes that an entity’s meaning is invariant. In reality, entities are polysemous. A person like "Nicole Kidman" might participate in a born_in relation as a citizen and an acted_in relation as an actress. As Miao Peng and Ben Liu argue in SMiLE, assigning a single static vector fails to capture these diverse semantic roles, leading to "representation rigidity" that hampers link prediction.
To solve this, a new generation of research has moved toward contextualized representations, where the embedding of an entity or relation is dynamically computed based on its immediate environment.
Relation-Specific Projections
The first step away from static vectors was to make entity representations dependent on the relation being queried. In TransH, Zhen Wang and colleagues introduced the idea of projecting entity embeddings onto a relation-specific hyperplane. This allows the same entity to have different "shadows" or incarnations depending on the context of the relationship.
Later models refined this using more sophisticated gating and attention mechanisms. TransGate, proposed by Jun Yuan, uses LSTM-style gates to filter which features of an entity are relevant to a specific relation. Similarly, AutoETER by Guanglin Niu uses relation-aware attention to prioritize specific latent "types" or attributes of an entity that justify a particular connection.
Dynamic Relations and Pairwise Interaction
Contextualization isn't just for entities; relations themselves can shift in meaning. Zequn Sun and Wei Hu introduced TransEdge, which contextualizes relation embeddings based on the specific head-tail entity pairs they connect. The intuition is that the "acted in" relation might imply different semantic nuances when connecting a "voice actor" to an "animated film" versus a "lead" to a "drama."
This idea of mutual influence is taken further by InterHT. Baoxin Wang suggests that head and tail entities should not be represented independently. Instead, they should interact contextually before a distance is even computed. This allows the model to disambiguate an entity’s role by looking at its "interaction partner."
Borrowing Context from Text and Structure
If a knowledge graph is sparse, there may not be enough internal structure to provide context. Researchers have turned to external text corpora to bridge this gap. TEKE, developed by Juanzi Li and Zhigang Wang, uses entity co-occurrence in text to dynamically adjust embeddings. By looking at how entities are mentioned in natural language, the model can infer which semantic facets are currently active.
The most recent trend involves treating the knowledge graph like a sequence of tokens in a language model. CoKE, by Haifeng Wang, uses a Transformer encoder to generate embeddings that are sensitive to the local graph neighborhood. Just as BERT represents a word based on the words around it, CoKE represents an entity based on the paths and neighbors surrounding it.
This structural contextualization is further refined in HittER by Sanxing Chen and Xiaodong Liu, which uses a hierarchical Transformer to aggregate information from an entity's local subgraph. By treating the graph as a rich, multi-layered context rather than a collection of isolated facts, these models can resolve the ambiguity that plagues static embedding methods.