Robust Question Answering against Distribution Shifts with Test-Time Adaptation: An Empirical Study

Hai Ye, Yuyang Ding, Juntao Li, Hwee Tou NgarXiv 2023 · DOI · arXiv

TL;DR

Question Answering (QA) models often crumble when faced with real-world data that differs from their training set. This paper introduces COLDQA, a comprehensive benchmark for evaluating model robustness, and proposes Online Imitation Learning (OIL), a novel test-time adaptation method that allows models to "learn on the fly" after deployment, significantly boosting performance against unexpected distribution shifts.

What Problem Does This Paper Address?

Machine learning models typically assume that training and test data come from the same distribution. In reality, a deployed QA system might encounter text corruption (typos, speech-to-text errors), different languages, or entirely new domains (e.g., moving from Wikipedia to medical texts).

While researchers often use Robustness Tuning (RT)—techniques like adversarial training applied before deployment—these methods can't anticipate every possible shift. The authors explore Test-Time Adaptation (TTA): can we improve a model after it has been deployed by letting it adapt to the specific stream of data it is currently seeing?

Methods

The authors focus on "online" TTA, where a model receives a stream of unlabeled test instances, makes a prediction, and then immediately updates its internal parameters before seeing the next instance.

Illustration of the test-time adaptation cycle: the model predicts on test data $x_t$, then updates itself to become $\pi_{t+1}$

Their core contribution is Online Imitation Learning (OIL). Standard TTA methods (like Tent or Pseudo-Labeling) can be unstable because they update the model based on its own potentially noisy predictions. OIL mitigates this by:

  1. Imitation Learning: Maintaining an "expert" model (a stable version of the source model) that guides a "learner" model. The learner tries to clone the expert's behavior while adapting to the new data.
  2. Causal Inference: Using a causal graph to identify and remove "model bias"—the tendency of the source model to force its old, potentially irrelevant knowledge onto the new distribution.
  3. Denoising: Filtering out high-loss (noisy) updates to prevent the model from learning from its own mistakes.
The proposed causal graph used to separate direct effects (input to output) from indirect model bias

Datasets

The authors introduce COLDQA, a unified benchmark that combines several existing datasets to test three types of shifts:

  • Text Corruption: NoiseQA (synthetic and natural noise).
  • Language Change: XQuAD and MLQA (cross-lingual transfer).
  • Domain Change: MRQA (including HotpotQA, NaturalQA, NewsQA, SearchQA, and TriviaQA).

The models are initially trained on SQuAD v1.1 (the source distribution) and then evaluated on the COLDQA subsets.

Key Insights and Ideas

  • TTA is a viable alternative to RT: Even without expensive pre-deployment adversarial training, simple TTA can achieve comparable robustness.
  • The "Better Together" Effect: The most significant performance gains come from combining both. Applying TTA to a model that has already undergone Robustness Tuning (RT+TTA) yields the best results.
  • Stability is King: Traditional TTA methods like Pseudo-Labeling (PL) are highly sensitive to hyperparameters (like learning rate). OIL is much more robust, maintaining performance across a wider range of settings.
  • Causal Intervention Works: By explicitly modeling and removing the "indirect effect" of model bias through causal inference, the authors show that the model adapts more cleanly to the target distribution.

Key Results

The experiments show that TTA consistently improves performance across all distribution shifts in COLDQA.

  • Performance Boost: On XLMR-large, the combination of RT (via xTune) and OIL improved the average Exact Match (EM) score by roughly 6 absolute points over the vanilla model.
  • Comparison: TTA alone is comparable to RT. For instance, on XLMR-base, TTA achieved an average EM of 57.06, while the best RT method (xTune) achieved 58.54.
  • Hyperparameter Robustness: OIL demonstrated significantly less variance than PL when changing learning rates and memory bank sizes, making it much safer for "set-it-and-forget-it" deployment.
Comparison of Exact Match (EM) scores for vanilla models, RT, TTA, and the combined RT+TTA approach

Limitations and Future Work

The authors acknowledge two primary hurdles for TTA in production:

  1. Computational Cost: Updating a large language model for every single test instance is computationally expensive and may increase latency.
  2. Hyperparameter Selection: In a real-world TTA scenario, you don't have labeled "gold" data to tune your learning rate or adaptation parameters. While OIL is more robust, finding the optimal settings without labels remains an open challenge.

Future work may focus on making these updates more efficient (e.g., updating only a small subset of parameters) and developing automated ways to tune TTA settings on the fly.

Summary updated

Abstract

A deployed question answering (QA) model can easily fail when the test data has a distribution shift compared to the training data. Robustness tuning (RT) methods have been widely studied to enhance model robustness against distribution shifts before model deployment. However, can we improve a model after deployment? To answer this question, we evaluate test-time adaptation (TTA) to improve a model after deployment. We first introduce COLDQA, a unified evaluation benchmark for robust QA against text corruption and changes in language and domain. We then evaluate previous TTA methods on COLDQA and compare them to RT methods. We also propose a novel TTA method called online imitation learning (OIL). Through extensive experiments, we find that TTA is comparable to RT methods, and applying TTA after RT can significantly boost the performance on COLDQA. Our proposed OIL improves TTA to be more robust to variation in hyper-parameters and test distributions over time.
BibTeX
@misc{ye2023robust,
  author = {Hai Ye and Yuyang Ding and Juntao Li and Hwee Tou Ng},
  title = {{Robust Question Answering against Distribution Shifts with Test-Time Adaptation: An Empirical Study}},
  year = {2023},
  eprint = {2302.04618},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2302.04618},
}
Figure