Abstract
Elad Segal, Avia Efrat, Mor Shoham, Amir Globerson, Jonathan Berant. Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP).
2020.
Introduction
Standard Reading Comprehension (RC) models, such as those based on BERT or RoBERTa, typically treat answer extraction as a "start-and-end" problem. They predict the most likely starting and ending tokens of a single contiguous span. While effective for datasets like SQuAD, this approach fails for questions where the answer is a list or a set of disconnected phrases (e.g., "Which two presidents were born in New York?").
Previous attempts to address this "multi-span" problem often involved complex, non-differentiable architectures or restricted the model to a fixed number of spans. This paper introduces TASE (TAg-based Span Extraction), a simple yet powerful shift that treats question answering as a sequence tagging task, allowing models to naturally extract multiple non-contiguous spans.
Methodology
Instead of predicting two indices, TASE predicts a tag for every token in the input using either BIO (Beginning, Inside, Outside) or IO (Inside, Outside) tagging schemes. This allows the model to pick and choose any number of relevant tokens from the text.
Key Innovations
- Marginal Likelihood Training: To handle cases where an answer string appears multiple times but only one instance is relevant, the model is trained by maximizing the marginal probability of all possible correct taggings. This allows the model to learn which occurrences are contextually relevant.
- Multi-Head Integration: To maintain performance on single-span questions, the authors combine the TASE head with a traditional Single-Span Extraction (SSE) head. A "manager" module dynamically decides which head's output to trust based on the input.
- Viterbi Decoding: For the BIO scheme, Viterbi decoding is used at test time to ensure the predicted sequence of tags is globally consistent (e.g., preventing an "I" tag from following an "O" tag).
Evaluation and Results
The model was evaluated on two challenging benchmarks: DROP (Discrete Reasoning Over Paragraphs), which requires numerical reasoning, and QUOREF (Question-Answering over Co-reference), where answers are frequently multi-span.
The TASE approach yielded significant improvements over previous state-of-the-art models. As shown in the results table, replacing a single-span architecture (SSE) with TASE improved performance on DROP span extraction questions by 9.9 Exact Match (EM) points (from 70.6 to 80.5). On QUOREF, the improvement was even more dramatic for the multi-span subset, jumping from 0.0 EM with the SSE head to nearly 60 EM with the integrated TASE heads.
Discussion and Qualitative Analysis
The authors found that simplicity is a key feature; casting QA as sequence tagging bypassed the need for complex symbolic programs. Interestingly, the simpler IO tagging performed nearly as well as BIO, suggesting that modern contextual embeddings like RoBERTa are powerful enough to distinguish span boundaries without explicit "Begin" tags.
Despite its success, the model has limitations. Qualitative analysis shows that the model occasionally splits a single long span into two shorter ones or predicts fewer spans than are present in the gold answer. Furthermore, while TASE excels at extraction, it does not solve the numerical reasoning challenges (like counting or arithmetic) inherent in the DROP dataset. If a question requires arithmetic not explicitly in the text, the model still struggles.
Conclusion
By reframing multi-span question answering as a sequence tagging task, the authors provide a differentiable and effective architecture that significantly outperforms traditional start-end pointers. The adoption of this multi-span head by subsequent top-performing models on the DROP leaderboard underscores its impact on the field.
Summary updated