ReaLM: Residual Quantization Bridging Knowledge Graph Embeddings and Large Language Models

Wenbin Guo, Xin Wang, Jiaoyan Chen, Lingbing Guo, Zhao Li, Zirui Chen

Abstract

Large Language Models (LLMs) have recently emerged as a powerful paradigm for Knowledge Graph Completion (KGC), offering strong reasoning and generalization capabilities beyond traditional embedding-based approaches. However, existing LLM-based methods often struggle to fully exploit structured semantic representations, as the continuous embedding space of pretrained KG models is fundamentally misaligned with the discrete token space of LLMs. This discrepancy hinders effective semantic transfer and limits their performance. To address this challenge, we propose ReaLM, a novel and effective framework that bridges the gap between KG embeddings and LLM tokenization through the mechanism of residual vector quantization. ReaLM discretizes pretrained KG embeddings into compact code sequences and integrates them as learnable tokens within the LLM vocabulary, enabling seamless fusion of symbolic and contextual knowledge. Furthermore, we incorporate ontology-guided class constraints to enforce semantic consistency, refining entity predictions based on class-level compatibility. Extensive experiments on two widely used benchmark datasets demonstrate that ReaLM achieves state-of-the-art performance, confirming its effectiveness in aligning structured knowledge with large-scale language models.

Introduction

Knowledge Graph Completion (KGC) is the task of predicting missing links in a structured knowledge base—for instance, identifying the spouse of a specific entity. While traditional methods utilize geometric embeddings (e.g., RotatE) to represent entities in a continuous vector space, they often lack the deep reasoning and linguistic context inherent in Large Language Models (LLMs). Conversely, LLMs operate in a discrete token space, creating a significant "impedance mismatch" when integrating structured graph data.

Existing attempts to bridge this gap face three primary hurdles: the fundamental mismatch between continuous vectors and discrete tokens, the "vocabulary explosion" caused by millions of unique entities, and the risk of hallucination when generating entity names as plain text.

Comparison of traditional LLM-KG integration vs. the proposed quantitative coding method.

Methodology: The ReaLM Framework

ReaLM (Residual Quantization Bridging Knowledge Graph Embeddings and Large Language Models) introduces a multi-stage pipeline to align these disparate spaces:

  1. Semantic Extraction: High-quality continuous embeddings are first generated for all entities using a traditional model like RotatE.
  2. Residual Vector Quantization (RVQ): This core component represents each entity as a sequence of 32 "codewords." By using a multi-stage process where each stage captures the residual error of the previous one, a small codebook (e.g., 1,000 tokens) can represent a vast number of unique entities without vocabulary explosion.
  3. Token Integration: These codewords are added to the LLM's vocabulary as learnable tokens. The model is then fine-tuned using Low-Rank Adaptation (LoRA) to process these sequences.
  4. Ontology-Guided Refinement: To ensure semantic consistency, ReaLM predicts the entity's class (e.g., "Person") alongside its codewords, using this as a filter to prevent type errors.
The complete ReaLM framework, illustrating KG embedding extraction, residual vector quantization, and LLM integration.

Experimental Evaluation

The framework was evaluated on standard benchmarks WN18RR (lexical relations) and FB15k-237 (real-world facts), including ontology-augmented versions (WN18RRO and FB15k-237O).

Key Results

  • Performance: ReaLM achieved state-of-the-art results, with an MRR of 0.608 on WN18RR, representing a 1.6%–3.5% improvement over previous LLM-based baselines.
  • Precision: The model demonstrated high accuracy in exact entity prediction (Hits@1), reaching 0.575 on WN18RR.
  • Efficiency: By utilizing RVQ, ReaLM successfully represented over 40,000 entities using only 1,000 additional tokens, maintaining computational efficiency.

Conclusion and Future Work

ReaLM effectively bridges the gap between continuous KG embeddings and discrete LLM tokens. By compressing semantic meaning into hierarchical codewords, it enables LLMs to reason over structured knowledge with high precision. Future work will focus on reducing the computational cost of training—currently requiring significant GPU hours—and extending the framework to dynamic, industrial-scale knowledge graphs.

Summary updated

Abstract

Large Language Models (LLMs) have recently emerged as a powerful paradigm for Knowledge Graph Completion (KGC), offering strong reasoning and generalization capabilities beyond traditional embedding-based approaches. However, existing LLM-based methods often struggle to fully exploit structured semantic representations, as the continuous embedding space of pretrained KG models is fundamentally misaligned with the discrete token space of LLMs. This discrepancy hinders effective semantic transfer and limits their performance. To address this challenge, we propose ReaLM, a novel and effective framework that bridges the gap between KG embeddings and LLM tokenization through the mechanism of residual vector quantization. ReaLM discretizes pretrained KG embeddings into compact code sequences and integrates them as learnable tokens within the LLM vocabulary, enabling seamless fusion of symbolic and contextual knowledge. Furthermore, we incorporate ontology-guided class constraints to enforce semantic consistency, refining entity predictions based on class-level compatibility. Extensive experiments on two widely used benchmark datasets demonstrate that ReaLM achieves state-of-the-art performance, confirming its effectiveness in aligning structured knowledge with large-scale language models.
BibTeX
@inproceedings{guo2026realm,
  author = {Wenbin Guo and Xin Wang and Jiaoyan Chen and Lingbing Guo and Zhao Li and Zirui Chen},
  title = {{ReaLM: Residual Quantization Bridging Knowledge Graph Embeddings and Large Language Models}},
  booktitle = {The Web Conference},
  year = {2026},
  doi = {10.1145/3774904.3792437},
}
Figure