Multi-Span Extraction for Non-Contiguous Question Answering

51 concepts40 papers

Standard extractive question-answering (QA) models operate on a simple assumption: the answer to a query exists as a single, continuous block of text. This design pattern, popularized by early reading comprehension benchmarks, relies on pointer networks to predict exactly two positions: the start token and the end token of the answer span.

While mathematically clean, this single-span constraint fails when applied to real-world information needs. If a user asks for a list of items, or if the evidence required to answer a conditional question is scattered across different paragraphs, a single continuous span cannot capture the answer cleanly. Forcing a model into this paradigm creates a dilemma: it must either extract a massive, noisy block of text containing irrelevant intermediate sentences, or truncate the answer and omit critical context.

To overcome this limitation, researchers have shifted toward multi-span extraction, developing architectures capable of identifying and aggregating non-contiguous segments of text into a single coherent response.

Reframing QA as Sequence Tagging

The foundational shift from single-span to multi-span extraction requires changing how the model outputs its predictions. Instead of treating the document as a single distribution from which to pick a start and end point, the problem is reframed as a sequence tagging task.

Document:  The  company  opened  offices  in  Paris  ,  London  ,  and  Tokyo .
Pointers:  [                               Start                       End ]
Tagging:    O      O        O       O     O   B-Ans O   B-Ans   O    B-Ans O

In this framework, models typically use an IO (Inside, Outside) or BIO (Begin, Inside, Outside) tagging scheme. For every token in the document, the model makes a distinct classification decision: is this token part of an answer span, or is it not?

This architectural shift solves the output flexibility problem. As demonstrated by Segal et al., treating reading comprehension as sequence tagging allows a model to naturally extract an arbitrary number of disconnected phrases or entities in a single pass. This approach yields significant performance gains on complex datasets like DROP and QUOREF, which require discrete reasoning and coreference resolution over scattered text.

Similarly, in specialized domains like biomedicine where answers are frequently list-based, Yoon et al. showed that sequence tagging eliminates the need for arbitrary confidence thresholds to determine how many answers exist, allowing the model to dynamically scale its predictions based on the text.

Stitching Together Fragmented Evidence

While sequence tagging solves the formatting challenge of non-contiguous QA, it introduces a deeper semantic challenge: independent token-level decisions lack a global understanding of how those tokens relate to one another. If a model tags two distant sentences as answers, how does it ensure they are part of the same logical reasoning chain?

To bridge these gaps, researchers introduce structural and global constraints into the network:

  • Global Span Counting: To prevent the model from over-extracting or under-extracting disjointed segments, Li et al. combined sequence tagging with a global prediction head. This head explicitly predicts the total number of expected spans in the document, acting as a regularizer that constrains token-level choices.
  • Structural Graphs: Rather than treating the document as a linear sequence of tokens, some frameworks explicitly map out its layout. Wang et al. introduced Hierarchical Graph Neural Networks (HGNNs) to connect sentences and paragraphs as nodes. By passing messages across these structural edges, the model can synthesize information from non-consecutive sections before making extraction decisions.
  • Sparsified Long-Range Attention: When documents are long, the linear distance between relevant facts weakens standard transformer attention. To combat this, Zhu utilized sparsified attention mechanisms designed to explicitly link distant, non-contiguous sentences, allowing the model to bypass intermediate noise and aggregate scattered evidence into a unified representation.

Where Multi-Span Extraction Matters

The necessity of retrieving non-contiguous answers spans across various domains where information is inherently unstructured or conversational:

  • Dialogue and Chat Logs: In customer service or multi-turn conversations, answers are rarely contained in a single utterance. As noted by Yang et al., complete answers are often fragmented across multiple dialogue turns, making multi-span extraction vital for reconstructing coherent FAQ databases from raw chat logs.
  • Clinical Reports: Medical narratives, such as radiological findings, are notoriously weak in structure. Dada et al. highlighted that critical clinical evidence is frequently distributed across disparate sentences in a report, meaning single-span constraints inherently truncate the full clinical picture.
  • Web Content and Clickbait Spoiling: Online text often distributes answers across paragraphs to maintain reader engagement. For example, Hagen et al. demonstrated that automatically "spoiling" clickbait headlines requires gathering multiple discrete items scattered throughout an entire article, a task that traditional extractive models fail to execute cleanly.

By moving away from the rigid assumption of contiguity, multi-span extraction aligns machine reading comprehension with the messy, distributed reality of human text.

Key Foundational Work

NewsQA: A Machine Comprehension Dataset (2016) — contributed 1 concept elements to this direction.
Overview of the NTCIR-11 Recognizing Inference in TExt and Validation (RITE-VAL) Task (2014) — contributed 1 concept elements to this direction.
History & citationVersion 2 · 2026-10-04

Current · Permanent link

  1. Version 2 (latest) · 2026-10-04 · 40 papers
  2. Version 1 · 2026-10-04 · 57 papers
Cite this direction

Latest version (a living page):

@misc{lacuna_non_contiguous_answer_retrieval_99824d7c,
  title = {non-contiguous answer retrieval},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/non-contiguous-answer-retrieval/rdr_99824d7ce4d3437e9e39544573c09dfa},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-21},
}

This exact version (version 2, frozen text):

@misc{lacuna_non_contiguous_answer_retrieval_99824d7c_v2,
  title = {non-contiguous answer retrieval},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/non-contiguous-answer-retrieval/rdr_99824d7ce4d3437e9e39544573c09dfa/v/2},
  note = {Version 2, frozen 2026-10-04. Accessed 2026-11-21},
}

See something wrong? Report a mistake