Abstract
KBQA is a system designed to bridge the gap between natural language questions and structured knowledge bases like Freebase. By learning over 1.1 million question templates from massive web-scale QA corpora (Yahoo! Answers), it can accurately map diverse human queries to specific database predicates, enabling precise answers to both simple and complex factoid questions. This paper details the two-phase architecture—offline template extraction and online question answering—and demonstrates how the system achieves an order of magnitude higher predicate coverage compared to traditional bootstrapping methods.
Introduction
Knowledge Bases (KBs) like Freebase contain billions of facts, but querying them usually requires specialized languages like SPARQL. For a regular user to ask a simple question like "How many people live in Honolulu?", a system must perform semantic parsing—translating that natural language into a structured query like (Honolulu, location.population, ?x).
The core challenge is the semantic gap: there are infinite ways to ask the same question (e.g., "What is the population of $city?" vs. "How many people are there in $city?"). Previous methods often relied on manual rules or small-scale "bootstrapping" techniques that lacked the coverage to handle the sheer variety of human expression and the thousands of different relations stored in a modern KB.
Methodology
The authors propose a template-based approach that operates in two distinct phases: offline extraction and online answering.
1. Offline Template Extraction
Instead of hand-coding rules, KBQA "learns" from 41 million QA pairs from Yahoo! Answers. The system identifies entities in the question and their corresponding values in the answer, then looks up the relationship (predicate) between them in Freebase. To ensure generality, specific entities are replaced with concepts (e.g., "Honolulu" becomes $city) using Probase, a massive semantic network.
2. Online Question Answering
When a user submits a question, the system identifies the entity using Stanford NER and replaces it with its concept. It then matches the resulting template against its repository of 1.1 million learned patterns. Finally, it retrieves the answer from Freebase, handling complex KB structures where a relationship might be represented by a path (e.g., spouse -> name) rather than a single link.
Datasets
- Yahoo! Answers: A corpus of 41 million QA pairs used to harvest templates.
- Freebase: The target large-scale knowledge base containing the facts.
- Probase: A semantic network used for conceptualization (mapping entities to categories like city, author, or athlete).
- Comparison Baseline: The system was compared against BOA (Bootstrapping Ontology Alignment) patterns, derived from a corpus of 256 million sentences.
Key Results
The evaluation of KBQA demonstrates the power of scale in template learning. KBQA discovered over 4,600 distinct relations, an order of magnitude higher than the 283 identified by the BOA baseline.
Despite using a smaller initial corpus (41M QA pairs vs. 256M sentences for BOA), KBQA found significantly more templates and predicates. This suggests that QA pairs are a much "richer" source of training data for this task than raw sentences. Furthermore, the system successfully demonstrated the ability to decompose complex questions into multiple "Binary Factoid Questions" (BFQs) to retrieve multi-part answers.
Conclusion and Future Work
KBQA demonstrates that templates serve as an effective middle ground between rigid rules and unstructured search. By leveraging massive QA corpora and conceptualization via Probase, the system achieves high coverage and precision. However, the system's accuracy remains reliant on the performance of external tools like Stanford NER. Future work will focus on addressing the ambiguity inherent in certain question templates and extending the system's capabilities to handle more complex multi-hop reasoning or temporal questions.
Summary updated