Grow-and-Clip: Informative-yet-Concise Evidence Distillation for Answer Explanation

Yuyan Chen, Yanghua Xiao, Bang Liu

Introduction

Modern Question Answering (QA) systems, while highly accurate, often operate as "black boxes." They provide answers without transparently explaining the reasoning or the specific context that justifies them. While some systems attempt to provide "evidence" by highlighting relevant sentences, these extracts are frequently bloated with "noise"—irrelevant information that distracts users and can even degrade model performance. For instance, a sentence-level evidence for a question about a settlement's founding might contain 37 words, while only 16 are actually pertinent to the answer. The challenge lies in distilling evidence that is simultaneously informative, concise, and grammatically readable.

The GCED Framework

To address this, we introduce the Grow-and-Clip Evidence Distillation (GCED) algorithm. Unlike traditional methods that extract entire sentences, GCED employs a surgical, token-level approach to isolate the most relevant information. The framework operates through a five-step pipeline:

  1. Answer-oriented Sentences Extractor (ASE): Identifies the minimal set of sentences necessary to predict the correct answer.
  2. Question-relevant Words Selector (QWS): Pinpoints specific keywords within the context that directly relate to the query.
  3. Weighted Syntactic Parsing Tree Constructor (WSPTC): Generates a grammatical tree of the selected sentences, utilizing attention weights from a Pre-trained Language Model (e.g., RoBERTa) to weight word-to-word connections.
  4. Evidence Forest Constructor (EFC): Isolates subtrees containing the critical question and answer words.
  5. Optimal Evidence Distiller (OED): Executes the core "Grow-and-Clip" optimization strategy.
The five-step GCED pipeline for evidence distillation.

The Grow-and-Clip Strategy

The "Grow-and-Clip" phase treats evidence extraction as a tree-based optimization problem. The goal is to find a subtree that maximizes a Hybrid Score, which balances informativeness (F1 score), conciseness (reciprocal of length), and readability (language model perplexity).

  • Grow Phase: The algorithm iteratively connects disparate subtrees in the "evidence forest" by adding parent and sibling nodes. This process continues until a single, coherent, and readable tree structure is formed.
  • Clip Phase: Once a connected tree is established, the algorithm prunes redundant branches that do not contribute significantly to the Hybrid Score, ensuring the final output is as lean as possible without losing essential meaning.
Visualization of the tree-based Grow and Clip searching process.

Experimental Evaluation

The GCED algorithm was evaluated on the SQuAD (1.1 and 2.0) and TriviaQA benchmarks. The results demonstrate a significant "Less is More" effect:

  • Massive Compression: GCED reduced evidence word counts by an average of 78.5% on SQuAD and 87.2% on TriviaQA compared to full-sentence extraction.
  • Performance Gains: Replacing the original full context with distilled evidence actually improved downstream QA model performance. Exact Match (EM) scores increased by 3.5%–4.1% on SQuAD and by 18.2%–19.3% on TriviaQA, suggesting that removing textual noise helps models focus on relevant signals.
  • Human-Level Quality: Human evaluators rated the distilled evidence consistently high (above 0.75 on a 0-1 scale) for informativeness, conciseness, and readability.

Conclusion and Future Work

GCED provides a traceable, explainable path for QA systems, which is critical for high-stakes domains. However, limitations remain. The system can struggle with highly complex nested clauses, occasionally resulting in unreadable snippets. Furthermore, the model currently lacks external world knowledge; it cannot "fill in the blanks" for relationships not explicitly stated in the text. Future research will focus on integrating commonsense reasoning and improving handling of complex syntactic structures.

Summary updated

Abstract

Interpreting the predictions of existing Question Answering (QA) models is critical to many real-world intelligent applications, such as QA systems for healthcare, education, and finance. However, existing QA models lack interpretability and provide no feedback or explanation for end-users to help them understand why a specific prediction is the answer to a question. In this research, we argue that the evidences of an answer is critical to enhancing the interpretability of QA models. Unlike previous research that simply extracts several sentence(s) in the context as evidence, we are the first to explicitly define the concept of evidence as the supporting facts in a context which are informative, concise, and readable. Besides, we provide effective strategies to quantitatively measure the informativeness, conciseness and readability of evidence. Furthermore, we propose Grow-and-Clip Evidence Distillation (GCED) algorithm to extract evidences from the contexts by trade-off informativeness, conciseness, and readability. We conduct extensive experiments on the SQuAD and TriviaQA datasets with several baseline models to evaluate the effect of GCED on interpreting answers to questions. Human evaluation are also carried out to check the quality of distilled evidences. Experimental results show that automatic distilled evidences have human-like informativeness, conciseness and readability, which can enhance the interpretability of the answers to questions.
BibTeX
@inproceedings{chen2022grow,
  author = {Yuyan Chen and Yanghua Xiao and Bang Liu},
  title = {{Grow-and-Clip: Informative-yet-Concise Evidence Distillation for Answer Explanation}},
  booktitle = {ICDE},
  year = {2022},
  url = {https://openreview.net/forum?id=xF56q9F8Tn},
}
Figure