MuGER2: Multi-Granularity Evidence Retrieval and Reasoning for Hybrid Question Answering

Yingyao Wang, Junwei Bao, Chaoqun Duan, Youzheng Wu, Xiaodong He, Tiejun Zhao

Introduction

MuGER² (Multi-Granularity Evidence Retrieval and Reasoning) is a framework designed for Hybrid Question Answering (HQA), where answers are derived from both structured tables and unstructured linked text. By retrieving evidence at multiple granularities—columns, rows, cells, and links—and employing a specialized Evidence Selector, MuGER² achieves state-of-the-art performance, significantly outperforming previous models on the HybridQA benchmark.

Hybrid Question Answering requires models to reason over heterogeneous data sources. For instance, a table might list athletes, but the answer to a question about an athlete's hometown river might reside in a Wikipedia snippet linked to a table cell.

The Granularity Trade-off

The authors identify a fundamental Granularity Trade-off in HQA. Coarse-grained evidence (e.g., a whole table row) is easier for retrievers to find but contains significant noise, burdening the reasoning model. Conversely, fine-grained evidence (e.g., a specific cell or link) is ideal for reasoning but difficult to retrieve accurately from a large candidate pool.

The Granularity Trade-off: Retrieval Recall vs. Reasoning F1.

MuGER² addresses this by bridging the gap between coarse retrieval and fine-grained reasoning, optimizing for both high recall and high precision. As shown in the figure above, MuGER² sits at the "sweet spot" of the granularity curve, balancing the high recall of coarse retrieval with the high precision of fine-grained reasoning.

The MuGER² Framework

MuGER² operates in two distinct stages: multi-granularity retrieval and answer reasoning.

The MuGER² two-stage pipeline: Multi-Granularity Evidence Retrieval followed by Evidence Selection and Reasoning.

Stage 1: Multi-Granularity Evidence Retrieval

MuGER² uses a Unified Retriever based on BERT to score four types of evidence simultaneously: Columns, Rows, Cells, and Links.

  • Joint Training: Training on all granularities allows the model to learn inter-dependencies (e.g., a relevant cell is likely in a relevant row).
  • Contrastive Learning (CL): CL helps the model distinguish between similar candidates within the same granularity, improving retrieval sharpness by "pushing away" negative examples in the semantic space.

Stage 2: Answer Reasoning

The Evidence Selector (E-SEL) filters the retrieved candidates. It aggregates multi-granularity scores to calculate "Table" and "Passage" scores, determining if the answer is "In-Table" or "In-Passage."

  • Navigation: If the answer is in a passage, E-SEL "navigates" to the specific link, feeding only the relevant text to a Reading Comprehension (RC) model. This targeted approach drastically reduces input noise for the reader compared to models that process entire rows or tables.

Experimental Results

Evaluated on the HybridQA benchmark (62,682 instances), MuGER² sets a new performance standard. The "Large" version achieved 56.3% Exact Match (EM) and 66.2% F1, a substantial improvement over the previous baseline, Hybrider (43.8% EM, 50.6% F1).

Key findings include:

  • Ablation Study: Removing the Evidence Selector caused EM scores to drop by nearly 8%, highlighting the importance of evidence navigation.
  • Retrieval Recall: The unified retriever reached a top-1 recall of 98% for multi-granularity evidence, meaning the correct answer source was almost always in the model's top pick.

Conclusion

While MuGER² excels at text-based retrieval and reasoning, it faces challenges with Numerical Reasoning (e.g., counting or arithmetic). Future work will focus on integrating these capabilities to handle the "Compute" category of questions more effectively, which remains a primary limitation of the current architecture.

Summary updated

Abstract

Hybrid question answering (HQA) aims to answer questions over heterogeneous data, including tables and passages linked to table cells. The heterogeneous data can provide different granularity evidence to HQA models, e.t., column, row, cell, and link. Conventional HQA models usually retrieve coarse- or fine-grained evidence to reason the answer. Through comparison, we find that coarse-grained evidence is easier to retrieve but contributes less to the reasoner, while fine-grained evidence is the opposite. To preserve the advantage and eliminate the disadvantage of different granularity evidence, we propose MuGER$^2$, a Multi-Granularity Evidence Retrieval and Reasoning approach. In evidence retrieval, a unified retriever is designed to learn the multi-granularity evidence from the heterogeneous data. In answer reasoning, an evidence selector is proposed to navigate the fine-grained evidence for the answer reader based on the learned multi-granularity evidence. Experiment results on the HybridQA dataset show that MuGER$^2$ significantly boosts the HQA performance. Further ablation analysis verifies the effectiveness of both the retrieval and reasoning designs.
BibTeX
@inproceedings{wang2022muger,
  author = {Yingyao Wang and Junwei Bao and Chaoqun Duan and Youzheng Wu and Xiaodong He and Tiejun Zhao},
  title = {{MuGER2: Multi-Granularity Evidence Retrieval and Reasoning for Hybrid Question Answering}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2022},
  url = {https://openreview.net/forum?id=r_KNPNgL6q},
}

Loading related content...

Figure