Parameter Efficient Knowledge Graph Embeddings

29 concepts25 papers1 proposal

Knowledge Graphs (KGs) represent real-world facts using triplets of (head entity, relation, tail entity). Traditional Knowledge Graph Embedding (KGE) models operate by assigning a unique, continuous vector to every single entity and relation in the graph.

As a knowledge base scales to millions of entities, this lookup-table approach encounters a massive bottleneck: the model's parameter footprint scales linearly with the number of entities ($O(|E|)$). This memory explosion makes it nearly impossible to deploy high-performing KGE models on resource-constrained hardware like mobile or edge devices.

To solve this problem, parameter-efficient KGE research focuses on a core insight: the relational structure and semantic constraints of a knowledge graph contain significant redundancies. By exploiting these properties, models can drastically compress or entirely bypass entity-specific lookup tables without sacrificing predictive accuracy.

Exploiting the Asymmetry of Entities and Relations

In virtually all real-world knowledge graphs, the number of unique entities ($|E|$) vastly outnumbers the number of unique relations ($|R|$). A highly effective strategy for parameter efficiency is to shift the model's complexity away from the massive entity set and onto the compact relation set.

An early foundation for this was established by Antoine Bordes et al. in Translating Embeddings for Modeling Multi-relational Data. Their TransE model enforced a simple translational logic ($\mathbf{h} + \mathbf{r} \approx \mathbf{t}$), demonstrating that low-parameter geometric constraints could scale to millions of facts where complex tensor factorization methods failed.

To push this asymmetry further, Zili Zhou et al. proposed a method in On Completing Sparse Knowledge Base with Transitive Relation Embedding that focuses primarily on relation-level transitive patterns. By modeling relational paths rather than heavy entity-specific features, they reduced computational complexity from a dependency on the number of entities to a dependency on the number of relations, accelerating training times.

Smarter relational logic can also compensate for smaller entity representations. In TranS: Transition-based Knowledge Graph Embedding with Synthetic Relation Representation, Xuanyu Zhang et al. introduced a synthetic relation representation that dynamically interacts with anchor-based entity representations. This allows the model to handle complex, overlapping relationships using a fraction of the standard parameter budget.

Similarly, Shihui Yang et al. developed TransMS: Knowledge Graph Embedding for Complex Relations by Multidirectional Semantics, which replaces large projection matrices with a single learnable scalar parameter per triplet to control multidirectional semantic flow.

This structural simplification is also evident in geometric models. As shown by Aleksandar Pavlovic 0002 et al. in SpeedE: Euclidean Geometric Knowledge Graph Embedding Strikes Back, replacing relation-specific width vectors with a single shared parameter removes geometric redundancies, significantly cutting down inference time and parameter counts while retaining expressiveness.

Quantization and Compositional Codebooks

Instead of storing a unique continuous vector for every entity, compositional models represent entities using a tiny, shared "codebook" of discrete components.

In Knowledge Graph Embedding Compression, Mrinmaya Sachan introduced an autoencoder-style architecture that represents entities as sequences of discrete "KD codes" rather than continuous vectors. A non-linear decoder reconstructs the embeddings on the fly, achieving up to 1000x compression with negligible performance loss. Intuitively, instead of storing a massive matrix, the model stores a small set of shared features and a discrete "recipe" for each entity.

Traditional KGE:  [Entity Lookup Table: |E| x Dimension] -> Massive Memory

Compositional:    [Entity Discrete Codes] -> [Shared Codebook] -> [Decoder] -> Low Memory

Surprisingly, the logic used to map entities to these shared components does not need to be overly engineered. Jiaang Li et al. demonstrated in Random Entity Quantization for Parameter-Efficient Compositional Knowledge Graph Representation that randomly matching entities to codewords in a small codebook performs exceptionally well. The random assignment acts as a regularizer, making entity representations mathematically distinct in low-dimensional spaces without requiring complex matching logic.

Decoupling Graph Size via Universal Encoders and Text

The most direct way to eliminate the $O(|E|)$ bottleneck is to remove entity lookup tables entirely, shifting from transductive to inductive representation learning.

In Entity-Agnostic Representation Learning for Parameter-Efficient Knowledge Graph Embedding, Mingyang Chen 0002 et al. proposed a universal encoder framework. Instead of looking up a static vector, a shared network dynamically generates entity representations based on local graph structure, slashing total parameters by up to 90%.

External textual descriptions offer another powerful inductive bias. In JAKET: Joint Pre-training of Knowledge Graph and Language Understanding, Donghan Yu et al. developed a framework where language models and graph structures mutually enrich each other. By deriving entity embeddings from text descriptions rather than unique parameter IDs, the model size is completely decoupled from the graph size, allowing the system to generalize to entirely unseen entities.

Neil Veira et al. followed a similar intuition in Unsupervised Embedding Enhancements of Knowledge Graphs using Textual Associations, augmenting existing KGEs with unsupervised word embeddings to prove that parameter optimization and structure matter more than sheer parameter volume.

For downstream tasks like recommendation systems, the entity lookup table can be bypassed entirely by restructuring the graph. Liangwei Yang et al. showed in MetaKRec: Collaborative Meta-Knowledge Enhanced Recommender System that by translating a sparse KG into dense item-item edges based on semantic rules, one can eliminate individual entity embeddings (like specific authors or locations) entirely while maintaining high recommendation accuracy.

Dimension Lifting and Structural Pruning

If continuous embeddings must be used, parameter efficiency can still be achieved by keeping the base embeddings extremely narrow and expanding them dynamically, or by aggressively pruning redundant features.

In From Wide to Deep: Dimension Lifting Network for Parameter-efficient Knowledge Graph Embedding, Borui Cai et al. introduced LiftNet. Instead of storing wide 512-dimensional vectors, LiftNet stores tiny 16-dimensional base vectors for entities. When calculating triplet scores, a deep upsampling network "lifts" these narrow vectors into a higher-dimensional space. Because compression is focused entirely on the entity embeddings ($|E| \gg |R|$), this architecture slashes model size by up to 96.9% without losing expressive power.

Wide Embedding Layer (Traditional):  [Entity ID] -> [================== 512-d ==================]

LiftNet (Deep Upsampling):           [Entity ID] -> [== 16-d ==] -> [Deep Network] -> [512-d]

Alternatively, models can utilize specialized training and pruning regimes to enforce compactness:

Key Foundational Work

Translating Embeddings for Modeling Multi-relational Data (2013) — contributed 1 concept elements to this direction.
Type-Constrained Representation Learning in Knowledge Graphs (2015) — contributed 1 concept elements to this direction.
Efficient Learning of Entity and Predicate Embeddings for Link Prediction in Knowledge Graphs (2015) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_parameter_efficient_knowledge_graph_embe_4520758c,
  title = {parameter-efficient knowledge graph embedding},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/parameter-efficient-knowledge-graph-embedding/rdr_4520758c087641baa0a2e274b5524348},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_parameter_efficient_knowledge_graph_embe_4520758c_v1,
  title = {parameter-efficient knowledge graph embedding},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/parameter-efficient-knowledge-graph-embedding/rdr_4520758c087641baa0a2e274b5524348/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake