Abstract
Existing low-resource Knowledge Graph Question Answering (KGQA) methods rely heavily on Large Language Models (LLMs) for semantic parsing of natural language questions to their corresponding logical forms (LF). However, LLMs face practical bottlenecks due to high computational requirements, limited knowledge of specific LFs, and the scarcity of annotated data for new KGs. We propose NS-KGQA, a zero-shot neuro-symbolic approach that utilizes neural KG embeddings to model KG structure without additional data. Our framework extracts a link-prediction-based symbolic question subgraph and resolves it using a Symbolic Resolver powered by Dual KG Embeddings. Extensive experiments on complex benchmarks like KQA Pro demonstrate that NS-KGQA outperforms LLM-based zero-shot baselines by an average of 26%.
Introduction
Knowledge Graph Question Answering (KGQA) typically requires translating a natural language question into a formal logical form (like SPARQL). Current "low-resource" methods rely heavily on Large Language Models (LLMs) to perform this translation. However, this approach hits three major walls: resource intensity, grammar gaps in specific logical forms, and the "cold-start" problem when introducing new KGs without annotated data. Furthermore, complex questions involving numerical reasoning or multi-hop logic are notoriously difficult for LLMs to solve reliably in a zero-shot setting. NS-KGQA addresses these challenges by decomposing the problem into non-parametric modules that combine neural structural modeling with symbolic logic.
Methodology
The NS-KGQA framework is built upon three distinct modules that allow it to operate without KG-specific training data:
- Dual KG Embeddings (KGE): Unlike standard embeddings, this module learns separate embeddings for entities/concepts and attributes/literals (numbers, dates). This separation preserves the semantics of numerical data while modeling the KG's structure.
- Symbolic Question Subgraph Generation: Instead of a custom parser, the system uses an off-the-shelf Abstract Meaning Representation (AMR) parser. It identifies entities, concepts, and "Numerical Operation Phrases" (NOPs) like smallest or greater than to build a symbolic graph representing the question's logic.
- Symbolic Resolver: This execution engine traverses the symbolic subgraph. It uses neural KGE scoring functions for link prediction (structural steps) and symbolic operators (via libraries like SymPy) for logical and arithmetic steps.
This neuro-symbolic fusion allows the neural side to handle the incomplete nature of KGs via link prediction, while the symbolic side handles precise logic and arithmetic.
Experimental Results
The model was evaluated on three major benchmarks: KQA Pro (compositional and numerical reasoning), MetaQA (multi-hop reasoning), and WebQSP (complex graph structures).
On the KQA Pro dataset, NS-KGQA achieved an accuracy of 53.04%, significantly outperforming the best zero-shot LLM baseline, FlexKBQA (28.48%), and even surpassing GPT-4 (37.43%). In MetaQA, it established a new zero-shot state-of-the-art by beating the previous best model by 31%. The system also demonstrated robustness by handling up to 10-hop reasoning, a significant challenge for traditional semantic parsers. Beyond accuracy, the system is highly efficient; once embeddings are learned, inference runs on a CPU with a latency of just 2 seconds per question.
Discussion and Conclusion
NS-KGQA demonstrates that a neuro-symbolic approach can effectively bridge the gap between incomplete KGs and precise logical reasoning in a zero-shot environment. By using AMR as a universal bridge and Dual KG Embeddings for structural prediction, the system achieves superior performance and efficiency compared to much larger LLMs. Future work will focus on improving the sensitivity of the AMR parser and refining numerical extraction tools, as parsing errors remain the primary bottleneck to even higher performance.
Summary updated