Abstract: Yu Wang, V. Srinivasan, Hongxia Jin. Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies.
2022.
Introduction
Knowledge-Based Question Answering (KBQA) for multi-hop reasoning requires a system to navigate through multiple connected entities and relations within a knowledge base (KB) to arrive at an answer. Traditional approaches typically rely on a rigid "single ground truth path" training paradigm. However, real-world knowledge bases like Freebase are dense and redundant, often containing multiple valid reasoning paths to the same correct answer. Forcing a model to learn only one human-annotated path makes it brittle and labor-intensive to train.
MRP-QA (Multiple Reasoning Paths KBQA) introduces a fresh perspective by treating reasoning paths as latent variables. By marginalizing over multiple valid paths, the system achieves state-of-the-art performance using only answer-level supervision, eliminating the need for expensive, human-annotated "gold paths."
Methodology
The core innovation of MRP-QA is its treatment of the reasoning path as a latent variable. Instead of maximizing the probability of a single labeled path, the objective function sums the probabilities of all valid paths leading to the correct answer. This marginalization allows the model to reward any logical path that works, rather than being penalized for missing a specific human-chosen route.
The architecture employs a Gated Recurrent Unit (GRU) with an attention mechanism. Crucially, unlike standard models that primarily track relations, MRP-QA incorporates entity representations at each step of the reasoning chain. This provides the model with essential semantic context about its current location in the knowledge graph. To maintain a manageable search space, the system uses Depth First Search (DFS) for candidate path generation and an Expectation-Maximization (EM) style approach to dynamically select the most probable paths during training. Additionally, Point-wise Conditional Mutual Information (PMI) is used to penalize generic answers associated with high-degree entities (like "USA"), forcing the model to focus on the specific constraints of the question.
Experimental Results
The authors evaluated MRP-QA on three major benchmarks: WebQuestionsSP (WQSP), ComplexWebQuestion-1.1 (CWQ), and PathQuestion-Large (PQL). These datasets range from simple 1-2 hop questions to complex multi-constraint queries requiring up to 6 hops.
MRP-QA achieved impressive results, particularly among models using only weak (answer-level) supervision. On WQSP, it reached an F1 score of 74.9%, outperforming previous best models. On the more complex CWQ dataset, it achieved 49.9% F1, a significant jump over existing methods like TransferNet and NSM+h. On the PQL+ subset, it reached 98.0% accuracy. Ablation studies confirmed the importance of the model's design: removing entity features from the RNN resulted in a 2.1% drop in F1 score, proving that knowing "which entity" the model is at is crucial for multi-hop navigation.
Conclusion and Future Work
MRP-QA demonstrates that learning from multiple reasoning paths is more robust than following a single annotated path. By leveraging entity context and marginalization, the system achieves high performance with minimal supervision. While currently limited by the initial DFS search space (typically 3 hops), future work aims to scale the model to even deeper reasoning chains and integrate unstructured text to fill gaps in structured knowledge bases.
Summary updated