For years, the dominant paradigm in Machine Reading Comprehension (MRC) treated the problem as an exercise in highlighting. Given a question and a reference text, models were trained to predict a start and end token—effectively "pointing" to a specific text span that contained the answer. While highly effective for simple factoid queries, this extractive approach fundamentally struggles with the complexity of real-world information needs.
Humans do not merely highlight text when answering questions; they synthesize. They gather fragments of information from multiple sentences or documents, resolve ambiguous pronouns, eliminate redundancies, and compose a novel, coherent response. The transition from extractive span-selection to abstractive answer synthesis represents a major shift toward true reading comprehension.
The Limits of Pointing
To understand why abstractive synthesis is necessary, we must look at how real people interact with information. When analyzing real-world search engine logs, researchers quickly discovered a massive gap between what users expect and what extractive models can provide.
In the creation of the MS MARCO dataset, Nguyen et al. (MS MARCO: A Human Generated MAchine Reading COmprehension Dataset) demonstrated that human annotators naturally generate responses by fusing multiple retrieved passages rather than copying a single text span. Similarly, Liu et al. (DuReader: a Chinese Machine Reading Comprehension Dataset from Real-world Applications) observed that more than half of human-generated answers in practical search scenarios do not exist as literal snippets in the source text.
When a model is forced to rely purely on extraction, several linguistic failures occur:
- Information Fragmentation: The evidence required to answer a question (such as a "Why" or "How" query) is often distributed across multiple sentences or entirely different documents. An extractive model can only pick one snippet, leaving the answer incomplete.
- Coreference Blindness: Source documents are filled with local pronouns ("he", "it", "this company"). Extracting a sentence verbatim often strips away the necessary context, rendering the answer ambiguous or incomprehensible to the end user.
- Redundancy and Noise: Raw text contains parenthetical information, relative clauses, and conversational filler. Extractive systems preserve this noise, whereas human synthesis filters it out to maximize clarity.
Early benchmarks often masked these limitations. As Mark Yatskar revealed in a qualitative analysis (A Qualitative Comparison of CoQA, SQuAD 2.0 and QuAC), many datasets marketed as "abstractive" were actually extractive games in disguise, requiring only minor grammatical edits rather than true information synthesis. Real-world applications demand that models bridge this extractive-abstractive gap.
The Mechanics of Synthesis: Extract-then-Synthesize
Directly generating an abstractive answer from a massive corpus or a long document is computationally expensive and prone to hallucinations. To solve this, researchers developed a powerful two-stage framework: coarse extraction followed by fine synthesis.
[User Query] ──> [Stage 1: Extractor] ──> [Noisy Evidence Spans] ──> [Stage 2: Synthesizer] ──> [Coherent Answer]
This architecture mimics human cognitive behavior. When asked a question, you first skim the text to locate relevant passages (extraction), and then you rephrase and combine those passages into a final response (synthesis).
Tan et al. pioneered this approach with S-Net (S-Net: From Answer Extraction to Answer Synthesis for Machine Reading Comprehension). The extraction stage acts as a hard-attention mechanism, filtering out irrelevant text and narrowing the search space. The synthesis stage—typically a sequence-to-sequence neural network—then takes these disjointed evidence spans and fuses them into a well-formed sentence.
However, pure generation introduces a new vulnerability: the model might forget rare words, misspell named entities, or hallucinate facts. To anchor the synthesis in reality, abstractive models rely on a copy mechanism (such as CopyNet). As shown by Takamura et al. (Extractive and Abstractive Summarization for Multiple-sentence Questions), a copy mechanism allows the network to dynamically choose between generating a new word from its vocabulary or copying a literal token directly from the source text. This hybrid approach ensures the final output remains factually grounded while enjoying the structural freedom of abstractive prose.
Resolving Context and Fusing Information
The true power of abstractive synthesis lies in its ability to perform linguistic cleanup and multi-hop reasoning.
Consider the challenge of pronoun resolution. In community forums or lengthy narratives, a user's core question might be buried under anecdotes and ambiguous references. Takamura et al. (Summarizing Lengthy Questions) demonstrated that abstractive models excel at question summarization because they can synthesize information across multiple sentences to replace generic pronouns with their specific, intended subjects.
This capacity for information fusion is even more critical in multi-hop reasoning, where a system must connect disparate clues to verify a claim or formulate an explanation. Ma et al. (EX-FEVER: A Dataset for Multi-hop Explainable Fact Verification) found that abstractive summarization provides vastly superior reasoning paths compared to extractive methods. By stripping away redundancies and stitching together isolated facts, the abstractive approach creates a fluid narrative context that is significantly more intelligible to both human evaluators and downstream AI models.
Synthesis Across Specialized Domains
As abstractive answer synthesis has matured, its principles have been deployed across diverse and highly specialized domains where extraction is fundamentally insufficient:
- Explainable Fact-Checking: In public health, providing a simple "True" or "False" verdict is not enough to build trust. Neema Kotonya (Explainable Automated Fact-Checking for Public Health Claims) demonstrated that abstractive synthesis can ingest complex, technical medical evidence and distill it into a coherent, layperson-accessible explanation.
- Medical Question Answering: Medical queries require extreme precision and contextual awareness. Tran et al. (ViMedAQA: A Vietnamese Medical Abstractive Question-Answering Dataset and Findings of Large Language Model) highlighted that abstractive synthesis is vital for generating medically accurate, rephrased responses that summarize dense clinical knowledge for patients.
- Multimodal Document Reading: Real-world documents contain visual layouts, charts, and spatial structures. Kyosuke Nishida (VisualMRC: Machine Reading Comprehension on Document Images) extended abstractive synthesis to document images, proving that models must integrate both visual layout signals and textual content to synthesize natural, human-like answers.
By moving beyond the constraints of verbatim text selection, abstractive answer synthesis transforms machine reading comprehension from a task of simple retrieval into an act of genuine information processing.
Key Foundational Work
Towards Automated Related Work Summarization (2010) — contributed 1 concept elements to this direction.
Syntactic Simplification for Improving Content Selection in Multi-Document Summarization (2004) — contributed 1 concept elements to this direction.