A Zero-Shot Neuro-Symbolic Approach for Complex Knowledge Graph Question Answering

Prerna Agarwal, Srikanta Bedathur2025 · DOI

Abstract

Existing low-resource Knowledge Graph Question Answering (KGQA) methods rely heavily on Large Language Models (LLMs) for semantic parsing of natural language questions to their corresponding logical forms (LF). However, LLMs face practical bottlenecks due to high computational requirements, limited knowledge of specific LFs, and the scarcity of annotated data for new KGs. We propose NS-KGQA, a zero-shot neuro-symbolic approach that utilizes neural KG embeddings to model KG structure without additional data. Our framework extracts a link-prediction-based symbolic question subgraph and resolves it using a Symbolic Resolver powered by Dual KG Embeddings. Extensive experiments on complex benchmarks like KQA Pro demonstrate that NS-KGQA outperforms LLM-based zero-shot baselines by an average of 26%.

Introduction

Knowledge Graph Question Answering (KGQA) typically requires translating a natural language question into a formal logical form (like SPARQL). Current "low-resource" methods rely heavily on Large Language Models (LLMs) to perform this translation. However, this approach hits three major walls: resource intensity, grammar gaps in specific logical forms, and the "cold-start" problem when introducing new KGs without annotated data. Furthermore, complex questions involving numerical reasoning or multi-hop logic are notoriously difficult for LLMs to solve reliably in a zero-shot setting. NS-KGQA addresses these challenges by decomposing the problem into non-parametric modules that combine neural structural modeling with symbolic logic.

Methodology

The NS-KGQA framework is built upon three distinct modules that allow it to operate without KG-specific training data:

The NS-KGQA framework overview, showing the pipeline from AMR parsing to symbolic resolution using Dual KG Embeddings.
  1. Dual KG Embeddings (KGE): Unlike standard embeddings, this module learns separate embeddings for entities/concepts and attributes/literals (numbers, dates). This separation preserves the semantics of numerical data while modeling the KG's structure.
  2. Symbolic Question Subgraph Generation: Instead of a custom parser, the system uses an off-the-shelf Abstract Meaning Representation (AMR) parser. It identifies entities, concepts, and "Numerical Operation Phrases" (NOPs) like smallest or greater than to build a symbolic graph representing the question's logic.
  3. Symbolic Resolver: This execution engine traverses the symbolic subgraph. It uses neural KGE scoring functions for link prediction (structural steps) and symbolic operators (via libraries like SymPy) for logical and arithmetic steps.

This neuro-symbolic fusion allows the neural side to handle the incomplete nature of KGs via link prediction, while the symbolic side handles precise logic and arithmetic.

Experimental Results

The model was evaluated on three major benchmarks: KQA Pro (compositional and numerical reasoning), MetaQA (multi-hop reasoning), and WebQSP (complex graph structures).

Performance comparison on the KQA Pro dataset, where NS-KGQA significantly outperforms other zero-shot methods.

On the KQA Pro dataset, NS-KGQA achieved an accuracy of 53.04%, significantly outperforming the best zero-shot LLM baseline, FlexKBQA (28.48%), and even surpassing GPT-4 (37.43%). In MetaQA, it established a new zero-shot state-of-the-art by beating the previous best model by 31%. The system also demonstrated robustness by handling up to 10-hop reasoning, a significant challenge for traditional semantic parsers. Beyond accuracy, the system is highly efficient; once embeddings are learned, inference runs on a CPU with a latency of just 2 seconds per question.

Discussion and Conclusion

NS-KGQA demonstrates that a neuro-symbolic approach can effectively bridge the gap between incomplete KGs and precise logical reasoning in a zero-shot environment. By using AMR as a universal bridge and Dual KG Embeddings for structural prediction, the system achieves superior performance and efficiency compared to much larger LLMs. Future work will focus on improving the sensitivity of the AMR parser and refining numerical extraction tools, as parsing errors remain the primary bottleneck to even higher performance.

Summary updated

Abstract

Existing low-resource Knowledge Graph Question Answering (KGQA) methods rely heavily on Large Language Models (LLMs) for semantic parsing of natural language question to its corresponding logical form (LF) such as SPARQL, S-Expression, etc.However, LLMs becomes bottleneck for practical applications due to: (1) its high computational resource requirements; (2) limited knowledge of LLM about different LFs; (3) unavailability of lowresource annotated data for new KGs and settings.This motivates us to design a KGQA framework that can operate in a zero-shot setting without the need for additional resources.In this paper, we propose (NS-KGQA) 1 : a zeroshot neuro-symbolic approach based on neural KG embeddings that have demonstrated their ability to effectively model the structure of the KG without the need for additional data.We extract a link-prediction based symbolic question subgraph.We then propose a Symbolic Resolver that uses Dual KG Embeddings combined with a symbolic approach to resolve the symbolic question subgraph.Our extensive experiments on Complex KGQA benchmarks such as KQA Pro demonstrate the effectiveness of our approach.NS-KGQA outperforms all other LLM-based zero-shot baselines by 26% (avg).
BibTeX
@misc{agarwal2025zero,
  author = {Prerna Agarwal and Srikanta Bedathur},
  title = {{A Zero-Shot Neuro-Symbolic Approach for Complex Knowledge Graph Question Answering}},
  year = {2025},
  doi = {10.18653/v1/2025.findings-emnlp.617},
}
Figure