In traditional graph and knowledge representation learning, entities and relations are treated as static lookup tables. An entity—whether it represents a person, a city, or a concept—is mapped to a single fixed-point vector $\mathbf{e} \in \mathbb{R}^d$.
While computationally efficient, this "one-size-fits-all" approach introduces a severe representation bottleneck, as noted by Ismail Ilkan Ceylan et al. in BoxE: A Box Embedding Model for Knowledge Base Completion. Real-world entities are inherently multifaceted and polysemous. For example, the entity "New York" acts as a geographic location when paired with the relation locatedIn, but shifts to a political or financial entity when paired with governedBy or headquarteredIn. Forcing these distinct semantic roles into a single static vector causes context blindness, effectively averaging out the nuanced features required to make accurate downstream predictions.
To overcome this limitation, a prominent research direction focuses on learning context-dependent representations. Instead of treating embeddings as static points, these approaches dynamically reshape, project, or contextualize representations based on the immediate relational, textual, or structural environment.
Geometric Projections and Spaces
The earliest and most intuitive way to build context-dependency is through geometric transformations. If an entity possesses multiple facets, we can isolate the relevant features by projecting its static base vector into a subspace dedicated to the relation at hand.
Static Space Relation-Specific Hyperplanes
[Entity] -------> Projection -------> Hyperplane A (Political Facet)
\----> Projection -------> Hyperplane B (Geographic Facet)
Zhen Wang 0036 et al. pioneered this with TransH (Knowledge Graph Embedding by Translating on Hyperplanes), which projects entity embeddings onto a relation-specific hyperplane before performing a translation. This allows the same entity to take on different coordinates depending on the mapping properties (such as one-to-many or many-to-many relationships) of the relation.
Yankai Lin et al. extended this concept in TransR (Learning Entity and Relation Embeddings for Knowledge Graph Completion) by moving beyond hyperplanes to entirely distinct relation-specific semantic spaces. Rather than just shifting the entity within the same global space, a projection matrix transforms the entity vector into a space where only the attributes relevant to that specific relation are preserved. To make these transformations even more flexible, Jinfa Yang et al. demonstrated in Improving Knowledge Graph Embedding Using Affine Transformations of Entities Corresponding to Each Relation that applying relation-specific affine transformations—allowing entities to both shift and scale dynamically—significantly boosts the performance of simpler baseline models with minimal parameter overhead.
Textual and Query-Driven Contextualization
While geometric projections handle relational context within structured graphs, they do not inherently capture external linguistic context or the specific intent of a user's query.
When entities appear within text or multi-hop reasoning chains, their active roles must be computed on the fly. Bo An et al. addressed this by introducing a mutual attention mechanism between entity descriptions and relation mentions in Accurate Text-Enhanced Knowledge Graph Representation Learning. This attention mechanism generates "triple-specific" embeddings, ensuring that the textual context directly shapes how the entity and relation interact.
For tasks involving language models, injecting static facts often fails because the static embeddings cannot adapt to the immediate topic of discussion. To bridge this gap, Tianxiang Sun et al. proposed CoLAKE (CoLAKE: Contextualized Language and Knowledge Embedding), which integrates text and structured knowledge into a unified Word-Knowledge graph. By training on this joint structure, the model learns contextualized representations where an entity's embedding is dynamically modulated by the surrounding sentence.
Linguistic Context: "The actress starred in..."
│
▼
[Unified Word-Knowledge Graph] ──► Dynamically emphasizes the "Actress" facet
▲
│
Structured Fact: (Nicole Kidman, profession, Actress)
This dynamic adaptation is equally crucial for logical reasoning. When answering complex graph queries, standard models often rely on static representations that fail to prioritize the neighbors necessary to satisfy specific constraints. Wen Zhang et al. resolved this in Prompt-fused Framework for Inductive Logical Query Answering by fusing a Transformer-encoded "query prompt" with local and global graph context. This framework forces entity representations to be query-aware, tailoring the node features directly to the logic required by the user's question.
Dynamic Neighborhood Selection and Disentanglement
In graph neural networks (GNNs), a node's representation is typically computed by aggregating information from its local neighborhood. However, a "one-size-fits-all" aggregation scheme introduces a different kind of static limitation: it assumes every neighbor or relationship is equally relevant across all scenarios.
As Chenchen Ye et al. observe in Context-aware Event Forecasting via Graph Disentanglement, aggregating all interactions indiscriminately leads to semantic entanglement. Distinct behavioral patterns get averaged out, making it difficult for a model to specialize its predictions. The authors propose disentangling event graphs into context-specific sub-graphs, allowing the model to isolate and collaborate across distinct situational backgrounds.
┌──► Neighbor A (Professional Context) ──► Sub-graph A
Node ───┼──► Neighbor B (Social Context) ──► Sub-graph B
└──► Neighbor C (Financial Context) ──► Sub-graph C
To dynamically control the scope of this context, Qiaoyu Tan et al. developed the Personalized Subgraph Selector framework (Bring Your Own View: Graph Neural Networks for Link Prediction with Personalized Subgraph Selection). Recognizing that nodes possess multiple structural meanings depending on the specific edge being evaluated, their framework automatically identifies the optimal neighborhood depth for each individual relationship.
Complementing this structural filtering, single-author works have introduced specialized architectures to handle neighborhood aggregation dynamically. Sanxing Chen developed HittER (HittER: Hierarchical Transformers for Knowledge Graph Embeddings), a hierarchical Transformer architecture that uses a two-tier approach to aggregate local neighborhoods based explicitly on the relation being queried. Similarly, to capture the contextual shifts that occur across entire facts rather than isolated nodes, Valeria Fionda proposed Triple2Vec (Learning Triple Embeddings from Knowledge Graphs), a method that bypasses individual node aggregation entirely by directly learning embeddings for entire triples, effectively embedding the context of the relationship natively.
By shifting from static lookup tables to dynamic, context-aware computation graphs, these diverse methodologies ensure that entity and relation representations remain as flexible and multifaceted as the real-world data they model.
Key Foundational Work
Knowledge Graph Embedding by Translating on Hyperplanes (2014) — contributed 1 concept elements to this direction.
Learning Entity and Relation Embeddings for Knowledge Graph Completion (2015) — contributed 1 concept elements to this direction.