Improved Synthetic Training for Reading Comprehension

Yanda Chen, Md. Arafat Sultan, Vittorio CastelliarXiv 2020 · OpenReview

Abstract

Automatically generated synthetic training examples have been shown to improve performance in machine reading comprehension (MRC). Compared to human annotated gold standard data, synthetic training data has unique properties, such as high availability at the possible expense of quality. In view of such differences, in this paper, we explore novel applications of synthetic examples to MRC. Our proposed pre-training and knowledge distillation strategies show significant improvements over existing methods. In a particularly surprising discovery, we observe that synthetic distillation often yields students that can outperform the teacher model.

Introduction

Machine Reading Comprehension (MRC)—the ability of a model to answer questions based on a given text—usually requires massive amounts of human-annotated data, which is expensive and slow to produce. While we can now automatically generate millions of "synthetic" question-answer pairs using language models, this data is a double-edged sword: it is abundant but often noisy or low-quality.

This paper demonstrates that synthetic data is most effective for MRC when used strategically rather than just as bulk filler. By targeting a model's specific weaknesses during pre-training and using massive amounts of unfiltered synthetic data for knowledge distillation, the researchers produced a "super-student" (BERT-Base) that actually outperforms its own teacher (BERT-Large).

Methods

The authors propose two primary novel strategies to move beyond simple data augmentation:

  1. Targeted Synthetic Pre-training (Highest Error): Instead of training on a random sample of synthetic data, they first train a model on "gold" (human-annotated) data. They then use this model to score millions of synthetic examples, specifically selecting those where the model has the highest prediction loss—essentially the questions the model finds most difficult.
  2. Synthetic Distillation: In standard knowledge distillation, a small "student" model tries to mimic the "soft" probability outputs of a large "teacher" model. Usually, this is done on the limited gold dataset. Here, the authors perform distillation on up to 6 million synthetic examples.

Crucially, they found that for distillation, "raw" synthetic data is better than "filtered" data. While filtering for "roundtrip consistency" helps standard training, it reduces diversity. For distillation, the student benefits more from seeing the teacher's behavior across a wider, noisier range of inputs.

Key Insights

A central discovery of this work is the relationship between synthetic data difficulty and model performance. In many machine learning contexts, "curriculum learning" suggests starting with easy examples. This paper finds the opposite for synthetic pre-training: Harder is better. Training on the subset of data where the model is most "confused" provides the most significant performance boost.

F1 performance across synthetic bins of increasing loss, showing that harder examples generally yield better results.

Furthermore, the study suggests that diversity trumps precision in distillation. When a student is learning from a teacher, the "correctness" of the synthetic label matters less than the teacher's reaction to the input. Using "raw" (unfiltered) synthetic data provides a richer "map" of the teacher's knowledge. This leads to the "Super-Student Effect," where a BERT-Base student can surpass the F1 score of the BERT-Large teacher it was learning from, suggesting that the bottleneck in many models is the amount of diverse data they've seen rather than their parameter count.

Key Results

The study utilizes two major MRC benchmarks: SQuAD 2.0 (Wikipedia-based) and NewsQA (CNN-based). The results across these datasets demonstrate the efficacy of the proposed methods.

Performance comparison of various synthetic training strategies on SQuAD 2.0 and NewsQA.
  • Outperforming the Teacher: On SQuAD 2.0, the BERT-Base student achieved an 81.4 F1 score, surpassing the BERT-Large teacher's 81.1.
  • Massive Gains over Baselines: The best distilled student showed an improvement of +6.7 to +7.7 absolute F1 points over a BERT-Base model trained only on human-annotated gold data.
  • Efficiency of Targeted Training: As shown in the results table, using just the "hardest" 500k or 1M synthetic examples for pre-training was often more effective than using the entire 4-million-example collection.

Limitations and Future Work

While the final student model is small and fast, the process of distilling 6 million examples is computationally expensive. Additionally, the study focuses strictly on extractive MRC; future work could explore if these "highest error" and "raw distillation" strategies work as well for generative tasks. Finally, performance was still trending upward at 6 million examples, suggesting there may be an even higher "upper bound" for synthetic data absorption.

Summary updated

Abstract

Automatically generated synthetic training examples have been shown to improve performance in machine reading comprehension (MRC). Compared to human annotated gold standard data, synthetic training data has unique properties, such as high availability at the possible expense of quality. In view of such differences, in this paper, we explore novel applications of synthetic examples to MRC. Our proposed pre-training and knowledge distillation strategies show significant improvements over existing methods. In a particularly surprising discovery, we observe that synthetic distillation often yields students that can outperform the teacher model.
BibTeX
@misc{chen2020improved,
  author = {Yanda Chen and Md. Arafat Sultan and Vittorio Castelli},
  title = {{Improved Synthetic Training for Reading Comprehension}},
  year = {2020},
  eprint = {2010.12776},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2010.12776},
}
Figure