CompKBQA: Component-wise Task Decomposition for Knowledge Base Question Answering

Yuhang Tian, Dandan Song, Zhijing Wu, Yang Pan, Changzhi Zhou, Jun Yang, Hao Wang, Huipeng Ma, Chenhao Li, Luan Zhang2025 · DOI

Abstract

Yuhang Tian, Dandan Song, Zhijing Wu, Pan Yang, Changzhi Zhou, Jun Yang, Hao Wang, Huipeng Ma, Chenhao Li, Luan Zhang. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing.

2025.

Introduction

Knowledge Base Question Answering (KBQA) involves translating natural language questions into structured logical forms that a database can execute. While Large Language Models (LLMs) have improved this process, they suffer from persistent "hallucination" issues. CompKBQA introduces a "divide and conquer" framework that decomposes complex query generation into four manageable sub-tasks: skeleton generation, entity identification, relation retrieval, and final assembly.

The Challenge: Hallucinations in KBQA

Generating a complex query in a single "coarse-grained" step is difficult for LLMs, which often lack a deep understanding of specific database schemas. This leads to three primary error types:

  1. Skeleton Errors: The model generates a query with the wrong logical structure or nesting.
  2. Entity Errors: The model hallucinates non-existent entities or uses incorrect surface names.
  3. Relation Errors: The model suggests relations that do not exist in the Knowledge Base (KB) schema.
Figure 1: Examples of common errors in KBQA and the underlying challenges of coarse-grained task formulation.

The CompKBQA Framework

CompKBQA (Component-wise Task Decomposition) breaks the generation process into a pipeline of four fine-tuned stages, making the LLM "KB-aware."

  1. Skeleton Generation: The LLM generates a structural template where entities and relations are replaced by placeholders (e.g., [ENT], [REL]). This allows the model to focus on logic without getting bogged down by specific vocabulary.
  2. Topic Entity Generation: The model identifies the key surface names of entities mentioned in the question.
  3. Relevant Relations Generation ($R^3$): A specialized retriever ($R^3$), trained via contrastive learning, pulls candidate relations from the KB. The LLM then selects the most pertinent relations from this list.
  4. Logical Form Generation: The model integrates the skeleton, entities, and relations into a complete, executable logical form.
Figure 3: The CompKBQA pipeline, illustrating the four stages of logical form generation and the training process.

Experimental Results and Insights

The framework was evaluated on the WebQSP and CWQ benchmarks based on Freebase.

  • State-of-the-Art Performance: CompKBQA achieved an F1 score of 81.3% on WebQSP and 79.4% on CWQ, outperforming previous fine-tuning methods like ChatKBQA.
  • Error Reduction: Compared to baselines, CompKBQA reduced Entity Errors by 36.3% and Relation Errors by 14.6%.
  • Structure vs. Content: By separating skeleton generation from filling, the authors prove that structural logic is a distinct skill that LLMs can master more effectively when isolated from the noise of specific database entries.
  • KB-Awareness: The $R^3$ module acts as a bridge, feeding the LLM the specific vocabulary of the KB in real-time, which drastically reduces hallucinated relations.

Limitations and Future Work

CompKBQA's accuracy is tied to the completeness of the underlying Knowledge Base. Furthermore, the final step of "Entity Alignment" (mapping a name like "Nicolas Cage" to a specific database ID) remains a significant source of remaining errors. Future work should focus on integrating entity linking more deeply into the generation process.

Summary updated

BibTeX
@misc{tian2025compkbqa,
  author = {Yuhang Tian and Dandan Song and Zhijing Wu and Yang Pan and Changzhi Zhou and Jun Yang and Hao Wang and Huipeng Ma and Chenhao Li and Luan Zhang},
  title = {{CompKBQA: Component-wise Task Decomposition for Knowledge Base Question Answering}},
  year = {2025},
  doi = {10.18653/v1/2025.emnlp-main.16},
}

Loading related content...

Figure