Learning with Instance Bundles for Reading Comprehension

Dheeru Dua, Pradeep Dasigi, Sameer Singh, Matt Gardner

Abstract

When training most modern reading comprehension models, all the questions associated with a context are treated as being independent from each other. However, closely related questions and their corresponding answers are not independent, and leveraging these relationships could provide a strong supervision signal to a model. Drawing on ideas from contrastive estimation, we introduce several new supervision techniques that compare question-answer scores across multiple related instances. Specifically, we normalize these scores across various neighborhoods of closely contrasting questions and/or answers, adding another cross entropy loss term that is used in addition to traditional maximum likelihood estimation. Our techniques require bundles of related question-answer pairs, which we can either mine from within existing data or create using various automated heuristics. We empirically demonstrate the effectiveness of training with instance bundles on two datasets -- HotpotQA and ROPES -- showing up to 11% absolute gains in accuracy.

Introduction

Standard reading comprehension (RC) models typically operate under the assumption that data points are independent and identically distributed (i.i.d.). In practice, this means that if a single context is associated with multiple questions, the model treats each question-answer pair as an isolated task.

However, questions related to the same context are rarely independent. For instance, questions like "Which city is more populous, Tokyo or New York?" and "Which city is less populous?" are logically linked. Humans naturally use these contrasts to learn nuances. By ignoring these relationships, models often fail to capture "distinguishing cues"—the subtle lexical changes that flip an answer. This leads to brittle performance where a model might correctly answer one question but fail its logical counterpart.

Training with Instance Bundles

To address this, we propose training on Instance Bundles: sets of closely related question-answer pairs derived from the same context.

Contrastive Estimation (CE)

While standard training uses Maximum Likelihood Estimation (MLE) on individual examples, we introduce Contrastive Estimation to normalize model scores over a "neighborhood" of related instances. We explore several strategies for this normalization:

  • Answer Conditional: Given a question, the model must distinguish the correct answer from other plausible answers in the bundle.
  • Question Conditional: Given an answer, the model identifies which question in the bundle it belongs to.
  • Joint/Full Partition: The model jointly optimizes the probability of all correct QA pairs within the bundle.
Visual representation of contrastive estimation strategies: (a) Answer Conditional, (b) Question Conditional, (c) Full partition, and (d) Joint.

The final training objective combines standard MLE with this new CE loss: $L = \alpha_1 L_{MLE} + \alpha_2 L_{CE}$.

Bundle Creation

Since most datasets lack explicit bundles, we employ three heuristics to generate them:

  1. Question Mining: Clustering existing questions with high lexical overlap.
  2. Question Generation: Using rule-based heuristics to flip superlatives (e.g., "taller" to "shorter") or negate verbs.
  3. Diverse Top-k Sampling: Generating multiple plausible but incorrect answer candidates using a pre-trained model.

Experimental Results

We evaluated our approach on HotpotQA (Comparison subset) and ROPES. The results demonstrate significant improvements across the board.

On the ROPES dataset, a T5-large baseline improved from 65.7% to 76.6% Exact Match (EM)—a 10.9% absolute gain. In HotpotQA, accuracy (F1) rose from 77.7 to 84.7. More importantly, consistency—defined as getting every question in a bundle correct—skyrocketed from 51.2% to 87.3%.

Distribution of normalized scores for different training methods. Contrastive Estimation (CE) significantly increases the model's confidence in distinguishing correct question-answer pairs.

Key Insights

The primary takeaway is the power of contrast. Models learn more effectively by seeing what an answer is not in relation to similar questions. By forcing the model to distinguish between "more" and "less" in the same gradient step, CE acts as a powerful regularizer that encourages logical consistency.

Interestingly, data augmentation alone (adding flipped questions to the training set with standard MLE) is insufficient. The contrastive objective is necessary to push the model toward true understanding. Furthermore, utilizing the bundle structure during inference (jointly deciding answers) can push performance even higher, reaching 90.1 F1 on HotpotQA.

Limitations and Conclusion

The effectiveness of this method depends heavily on the quality of the bundles. When distractors are too dissimilar or ungrammatical, the contrastive signal weakens. Future research should focus on more robust, automated methods for generating high-quality "hard" bundles for diverse datasets. By moving beyond i.i.d. training, we can build RC models that are not only more accurate but also more logically consistent.

Summary updated

Abstract

When training most modern reading comprehension models, all the questions associated with a context are treated as being independent from each other. However, closely related questions and their corresponding answers are not independent, and leveraging these relationships could provide a strong supervision signal to a model. Drawing on ideas from contrastive estimation, we introduce several new supervision techniques that compare question-answer scores across multiple related instances. Specifically, we normalize these scores across various neighborhoods of closely contrasting questions and/or answers, adding another cross entropy loss term that is used in addition to traditional maximum likelihood estimation. Our techniques require bundles of related question-answer pairs, which we can either mine from within existing data or create using various automated heuristics. We empirically demonstrate the effectiveness of training with instance bundles on two datasets -- HotpotQA and ROPES -- showing up to 11% absolute gains in accuracy.
BibTeX
@inproceedings{dua2021learning,
  author = {Dheeru Dua and Pradeep Dasigi and Sameer Singh and Matt Gardner},
  title = {{Learning with Instance Bundles for Reading Comprehension}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2021},
  doi = {10.18653/v1/2021.emnlp-main.584},
}
Figure