Contextualized Representations for Relational Graphs

This direction is no longer present in the current build. Its last version stays readable and citable here.

Version 1 of 176 papers105 conceptsfrozen 2026-10-04

In traditional network and knowledge graph embedding, every entity and relation is assigned a single, static vector. This "one-size-fits-all" approach assumes that an entity’s identity is invariant, regardless of who it is interacting with. However, real-world entities are multi-faceted. A node representing "Apple" in a knowledge graph performs a different semantic role when connected to "Newton" than when connected to "iPhone."

A growing body of research moves away from these static lookup tables toward contextualized representations. In this paradigm, the vector for a node or edge is dynamically computed based on its local neighborhood, the specific partner it is interacting with, or the broader task context.

The Problem of Static Identity

Early models like TransE treated relations as simple translations in a vector space. While efficient, these models struggle with complex mapping properties like 1-to-N or N-to-N relationships. If "London" is the capital of "UK" and also a "location" for "The Olympics," a static model forces "UK" and "The Olympics" to occupy similar coordinates, leading to semantic collapse.

To solve this, Guoliang Ji and colleagues introduced TransD (Knowledge Graph Embedding via Dynamic Mapping Matrix). Instead of a fixed matrix for each relation, TransD constructs a dynamic mapping matrix for every specific entity-relation pair. By using the outer product of projection vectors, the model ensures that the transformation applied to an entity is unique to the context of that specific triple.

Mutual Interaction and Attention

If entities are multi-faceted, their representation should "morph" depending on the specific neighbor they are currently "talking" to. Cunchao Tu and Zhiyuan Liu pioneered this with CANE (CANE: Context-Aware Network Embedding for Relation Modeling). CANE uses a mutual attention mechanism to let two nodes inspect each other's text descriptions. The resulting embedding is not a global property of the node, but a dynamic reflection of that specific pairwise interaction.

This concept of mutual contextualization is further refined in InterHT by Baoxin Wang and colleagues (InterHT: Knowledge Graph Embeddings by Interaction between Head and Tail Entities). They argue that head and tail entities should interact before the distance is computed. By weaving the relation embedding into this mutual contextualization, the model allows the semantic context of the relationship to influence how entities modify each other's representations, mirroring how word meanings shift based on surrounding syntax.

Relations as Dynamic Context

It isn't just entities that are context-dependent; relations themselves can change meaning. In biological networks, a "regulates" link might mean something very different depending on the specific proteins involved.

Zequn Sun and Wei Hu addressed this with TransEdge (TransEdge: Translating Relation-contextualized Embeddings for Knowledge Graphs). They treat relations as "edge-centric" translations that are contextualized by the head-tail pairs they connect. Similarly, in the medical domain, Zhe Yang and Junlan Feng demonstrated that treating relations as dynamic nodes within a neighborhood—rather than static labels—allows models to capture nuanced, domain-specific interactions (Learning to Leverage High-Order Medical Knowledge Graph for Joint Entity and Relation Extraction).

From Graphs to Sequences

A powerful trend in this area is the "Transformerization" of graphs. If we treat a path through a graph like a sentence, we can use self-attention to contextualize every element in that path.

CoKE, developed by Haifeng Wang and Wenbin Jiang, uses a Transformer encoder to generate dynamic embeddings for entities and relations based on their local graph context (CoKE: Contextualized Knowledge Graph Embedding). This allows the model to handle "entity polysemy" naturally: the same node ID will result in different vector outputs depending on the sequence of relations it appears in.

This idea extends to cross-modal tasks as well. Tianxiang Sun and Xipeng Qiu introduced CoLAKE, which integrates structured knowledge with raw text into a unified Word-Knowledge graph (CoLAKE: Contextualized Language and Knowledge Embedding). By jointly training on both, the model learns representations that adapt dynamically to the surrounding linguistic context, ensuring that the most relevant factual relationships are prioritized for a given sentence.

Shifting the Global to the Local

While dynamic models are expressive, they can be computationally expensive. A middle ground involves "shifting" static global embeddings into task-specific spaces.

Ping Wang and Khushbu Agarwal proposed SLiCE, which uses a translation layer to shift global node embeddings based on the local subgraph context (Self-Supervised Learning of Contextual Embeddings for Link Prediction in Heterogeneous Networks). This mechanism allows a single node to occupy different positions in the vector space depending on the specific community or relation type it is currently participating in, providing a balance between the stability of global features and the precision of local context.

Papers in this version

Showing 60 of 76 papers.

History & citationVersion 1 · 2026-10-04

No longer present in the current build · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_contextualized_representations_for_relat_0f13c029,
  title = {Contextualized Representations for Relational Graphs},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/contextualized-representations-for-relational-graphs/rdr_0f13c029e9164fa39df790bd5031254f},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-22},
}

This exact version (version 1, frozen text):

@misc{lacuna_contextualized_representations_for_relat_0f13c029_v1,
  title = {Contextualized Representations for Relational Graphs},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/contextualized-representations-for-relational-graphs/rdr_0f13c029e9164fa39df790bd5031254f/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-22},
}