Introduction
Open-domain question answering (QA) systems typically rely on two distinct types of knowledge sources: structured Knowledge Bases (KBs) like Wikidata and unstructured text corpora like Wikipedia. While KBs provide precise, structured facts, they often suffer from incompleteness. Conversely, text corpora offer massive coverage but are difficult to reason over algorithmically. Standard "retrieve-and-read" systems often treat retrieved passages in isolation, which leads to failure when an answer requires synthesizing information across multiple documents or disambiguating entities based on context.
This research introduces a novel framework that bridges these two worlds. By representing retrieved text passages as a graph connected by KB relations, the system can identify relevant evidence that keyword-based methods miss and "read" across multiple passages to solve complex queries. This approach demonstrates significant performance gains, outperforming standard text-only models by up to 11%.
The Challenge of Keyword Matching and Isolated Reading
Traditional QA systems face a "keyword matching" bottleneck. If a question does not share specific words with the ideal evidence passage, standard retrieval methods like TF-IDF often fail to find it. Furthermore, standard readers process passages independently, which can lead to errors in disambiguation. For example, a model might identify a person mentioned in a song's credits as the answer to a question about a singer, without realizing that a related passage about the album provides the necessary context to disqualify that candidate.
Methodology
The proposed framework consists of two primary components: GraphRetriever and GraphReader.
GraphRetriever
Instead of returning a flat list of passages, GraphRetriever constructs a structured graph. It begins with "seed" passages identified through traditional keyword search and entity linking. It then expands this set by traversing edges in a Knowledge Base (Wikidata) and following "inner-document" links (e.g., other passages within the same Wikipedia article). This expansion allows the system to discover highly relevant text that may not contain any direct keywords from the original query.
GraphReader
While most readers process each passage in a vacuum, GraphReader utilizes a BERT-based architecture enhanced with graph-aware fusion layers. These layers employ a Graph Convolutional Network (GCN) approach to propagate information between passages along the edges of the constructed graph. This ensures that the representation of any single passage is informed by its neighbors before a final answer is predicted.
Experimental Results
The model was evaluated on three major open-domain QA benchmarks: WebQuestions, Natural Questions, and TriviaQA. Using Wikipedia as the text source and Wikidata as the KB, the graph-based approach consistently outperformed non-graph baselines by 2-11% in Exact Match (EM) scores.
Key findings from the evaluation include:
- Early Fusion Benefits: Using the KB to guide retrieval allows the system to find evidence that keyword-based methods miss entirely.
- Structural Shortcuts: KB relations act as shortcuts between distant pieces of text. For instance, a "parent organization" link can connect a subsidiary's history to its parent company's founding date, even if the information resides in different Wikipedia articles.
- Relational Awareness: While even a simple binary indicator of a relationship between passages improves performance, explicitly modeling the specific type of relation (e.g., "performer" or "is located in") yields the best results.
Conclusion and Future Work
This work demonstrates that structured knowledge can significantly enhance the retrieval and reading of unstructured text for open-domain QA. By leveraging the connectivity of Knowledge Bases, the system achieves state-of-the-art results without the need for massive, end-to-end pre-training.
Future research may address current limitations, such as the memory intensity of training on full passage graphs and the dependency on the coverage of the underlying KB. Additionally, exploring joint end-to-end training of the retriever and reader could provide further performance improvements, albeit at a higher computational cost.
Summary updated