LSP-RAG: Latent Scoring Perspectives for Knowledge Graph Retrieval-Augmented Generation

Haochen Liu, Guanghui Min, Song Wang, Yada Zhu, Chen Chen, Jundong LiarXiv 2026 · arXiv

TL;DR

LSP-RAG is a novel framework that enables lightweight language models (like Flan-T5 or LLaMA-7B) to perform high-quality multi-hop retrieval from Knowledge Graphs (KGs). By using "latent scoring perspectives" guided by soft prompts, it captures diverse relational signals and maintains contextual continuity, outperforming even ChatGPT-based retrieval systems at a fraction of the computational cost.

What Problem Does This Paper Address?

While Large Language Models (LLMs) are impressive, they often "hallucinate" or rely on outdated information. Retrieval-Augmented Generation (RAG) fixes this by providing external evidence. Knowledge Graphs (KGs) are particularly useful because they represent facts as structured triplets (subject, relation, object), allowing for precise, multi-hop reasoning.

However, current KG-RAG methods face three major hurdles:

  1. Heterogeneous Signals: A single scoring metric (like semantic similarity) struggles to distinguish between different types of relevance, such as direct factual links versus temporal or indirect associations.
  2. Contextual Coherence: In multi-hop reasoning, a specific triplet might look irrelevant when judged in isolation but is actually a vital "bridge" connecting earlier steps to the final answer.
  3. Computational Cost: To solve the above, many state-of-the-art systems rely on massive models like ChatGPT to guide the retrieval process, which is expensive and slow for real-time applications.
Illustration of the two key challenges: Heterogeneous Signals and Contextual Coherence.

Methods

The authors introduce LSP-RAG (Latent Scoring Perspectives for RAG). Instead of fine-tuning the entire backbone model, they use soft prompting—learnable continuous embeddings prepended to the input—to adapt lightweight models for complex retrieval tasks.

The core of the framework is the Multi-Perspective Prompted Scoring module, which uses two types of soft prompts:

  • Perspective-Specific Soft Prompts: The model maintains $K$ different "perspectives." Each perspective uses a unique soft prompt to induce a distinct scoring function. This allows the model to look at a candidate triplet through multiple "lenses" simultaneously.
  • Contextual Soft Prompts: These prompts evolve with each reasoning hop. They encode the query and the history of previously explored triplets, ensuring the model maintains a "thread of thought" across multi-hop paths.

These perspectives are adaptively aggregated using a gating mechanism (similar to a Mixture-of-Experts approach) to produce a final relevance score for each candidate triplet.

Overview of the LSP-RAG framework, showing the iterative hop-wise retrieval and multi-perspective scoring.

Datasets

The researchers evaluated LSP-RAG on four established Question Answering (QA) benchmarks:

  • WebQuestionsSP (WebQSP): Natural language questions over the broad-coverage Freebase KG.
  • WikiMovies: A domain-specific dataset focused on movie-related facts.
  • MetaQA (1-hop & 2-hop): A more complex version of WikiMovies featuring paraphrased questions and multi-hop reasoning requirements.

Key Insights and Ideas

  • Latent vs. Explicit Roles: Rather than manually defining semantic roles (e.g., "find the date" or "find the actor"), the model learns latent perspectives. This flexibility allows it to adapt to the messy, diverse nature of real-world knowledge graphs.
  • Bridging the "Isolation" Gap: By using contextual soft prompts, the model avoids discarding "bridge" triplets that don't look immediately relevant to the query but are necessary for the next hop.
  • Small Models, Big Results: The paper demonstrates that with the right inductive biases (the latent perspectives), a 3B or 7B parameter model can rank triplets more accurately than a 175B+ parameter model using standard prompting.

Key Results

LSP-RAG consistently achieved the highest accuracy across all datasets and model backbones (LLaMA2-7B, LLaMA3-8B, and Flan-T5-3B).

  • Outperforming ChatGPT: Remarkably, LSP-RAG using a lightweight retriever outperformed systems like Think-on-Graph (ToG) and Reasoning on Graphs (RoG) even when those baselines used ChatGPT for retrieval. For example, on WebQSP, LSP-RAG (Flan-T5) reached 88.59% accuracy, significantly higher than ChatGPT-based ToG (74.24%).
  • Ablation Success: Removing either the perspective-specific or the contextual prompts led to a significant drop in performance, confirming that both "diverse lenses" and "memory" are essential for multi-hop KG retrieval.
  • Efficiency: Because it uses smaller models for the heavy lifting of retrieval, LSP-RAG offers a much more scalable and cost-effective solution for production environments.
Impact of the number of perspectives (K) on accuracy. Performance typically peaks or saturates around K=3.

Limitations and Future Work

  • Scaling with K: While more perspectives generally help, the computational cost increases linearly with the number of perspectives ($K$).
  • Fixed Hop Budget: The current implementation uses a fixed number of hops ($H$), which might miss extremely long or indirect reasoning paths.
  • KG Quality: The study focused on relatively clean, well-structured KGs. Testing the robustness of LSP-RAG on noisy, incomplete, or massive-scale heterogeneous graphs is a suggested next step.
  • Interpretability: While the retrieved triplets provide a clear reasoning trace, the internal "latent perspectives" themselves remain somewhat opaque and are not yet mapped to human-understandable semantic categories.

Summary updated

Abstract

Large Language Models (LLMs) have demonstrated strong capabilities in open-domain question answering, but often struggle with factual accuracy and multi-hop reasoning due to the incompleteness of the training corpora. A promising solution is Knowledge Graph Retrieval-Augmented Generation (KG-RAG), which supplements LLMs with structured knowledge retrieved from external knowledge graphs (KGs). However, existing KG-RAG methods either rely on large-scale language models (e.g., ChatGPT) to guide the retrieval process, which leads to high computational costs, or suffer from limited retrieval quality when using lightweight language models, particularly under multi-hop scenarios. We propose LSP-RAG (Latent Scoring Perspectives for Retrieval-Augmented Generation over Knowledge Graphs), a novel KG-RAG framework that enhances hop-wise KG knowledge retrieval through multi-perspective prompted scoring. Each latent scoring perspective is guided by a combination of two types of soft prompts: a perspective-specific soft prompt induces a distinct prompt-conditioned scoring function over candidate triplets, while a contextual soft prompt evolves with each reasoning hop by encoding the query and previously explored KG triplets, enabling the model to preserve consistency and relevance across multi-hop retrieval. Rather than assuming manually defined semantic roles, these latent perspectives are adaptively aggregated to capture heterogeneous relevance signals during retrieval. This design allows LSP-RAG to perform accurate and robust retrieval using lightweight language models. LSP-RAG achieves superior performance across multiple KG-based Question Answering benchmarks compared to existing retrieval systems, including those that rely on much larger language models, demonstrating its effectiveness under limited computational budgets.
BibTeX
@misc{liu2026lsp,
  author = {Haochen Liu and Guanghui Min and Song Wang and Yada Zhu and Chen Chen and Jundong Li},
  title = {{LSP-RAG: Latent Scoring Perspectives for Knowledge Graph Retrieval-Augmented Generation}},
  year = {2026},
  eprint = {2611.25229},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.25229},
}
Figure