On reducing sampling variance in covariate shift using control variates

Wouter M. Kouw, Marco LoogarXiv 2017 · arXiv · OpenReview

Abstract: Cross-validation under sample selection bias can, in principle, be done by importance-weighting the empirical risk. However, the importance-weighted risk estimator produces sub-optimal hyperparameter estimates in problem settings where large weights arise with high probability. We study its sampling variance as a function of the training data distribution and introduce a control variate to increase its robustness to problematically large weights.

Introduction

In many real-world machine learning applications, the training data (source domain) does not perfectly match the population intended for prediction (target domain). This phenomenon, known as sample selection bias or covariate shift, is common in fields like medicine or finance where data distributions shift over time or across different institutions.

The standard theoretical solution is importance weighting, which weights each training sample by the ratio of its probability in the target domain versus the source domain. While this approach removes bias in expectation, it is notoriously unstable. When a training point is rare in the source but common in the target, it receives an extremely large weight. These outliers cause the estimator's variance to explode, leading to poor hyperparameter selection during cross-validation and, ultimately, models that generalize poorly.

Methodology: The Controlled Importance-Weighted Risk Estimator

To address this instability, we propose leveraging control variates, a classic variance-reduction technique. The core idea is to use a correlated variable with a known expected value to "correct" the main estimate.

In the context of importance weighting, the weights $w(x)$ themselves serve as an ideal control variate because their expected value is known to be exactly 1. We modify the standard weighted risk estimator by subtracting a scaled version of the weights' deviation from their mean:

$$\hat{R}_\beta = \text{Weighted Loss} - \beta(w(x) - 1)$$

By deriving an optimal scaling factor $\beta^*$ that minimizes variance—estimated on the fly via simple least-squares regression—we can stabilize the risk estimate. This approach is computationally efficient as it utilizes the weights already calculated for the standard importance-weighting process.

Comparison of sampling variance for standard importance weighting ($\sigma^2_W$) and the proposed control variate estimator ($\sigma^2_\beta$) against the target variance ($\sigma^2_\mathcal{T}$).

Experimental Evaluation

We evaluated the proposed method using both synthetic and real-world datasets:

  • Synthetic Data: We utilized bivariate normal distributions where the source domain was a shifted and scaled version of the target. This allowed for precise control over the severity of the selection bias (represented by $\gamma$).
  • Ozone Level Detection: Using the UCI Ozone Level Detection dataset, we simulated selection bias by sampling the source domain locally in time using a Gaussian window, creating a natural distribution shift compared to the full dataset.

Key Results and Discussion

The controlled estimator demonstrated superior robustness compared to standard importance weighting. As shown in Figure 4, while the standard estimator's sampling variance ($\sigma^2_W$) increases dramatically as the distribution shift becomes more severe, the controlled version ($\sigma^2_\beta$) remains significantly more stable, tracking much closer to the target variance.

Mean target risk across varying source distribution parameters, demonstrating that the controlled estimator ($\hat{R}_{\hat{\beta}}$) consistently outperforms standard importance weighting ($\hat{R}_{\hat{w}}$).

In terms of predictive performance, Figure 6 illustrates that the mean target risk for the controlled estimator is consistently lower than that of the standard weighted estimator across various levels of source variance. On the Ozone dataset, the improvements were statistically significant, with $p$-values ranging from $10^{-30}$ to $10^{-50}$, indicating highly consistent variance reduction.

Conclusion

By using importance weights as a stabilizer rather than just a source of variance, we can make cross-validation significantly more reliable under covariate shift. The proposed controlled estimator provides a robust, computationally "cheap" drop-in replacement for standard importance weighting, offering a more production-ready pipeline for handling real-world distribution shifts. Future work may explore more complex control variates, such as higher-order moments of the weights, to further minimize variance.

Summary updated

Abstract

Cross-validation under sample selection bias can, in principle, be done by importance-weighting the empirical risk. However, the importance-weighted risk estimator produces sub-optimal hyperparameter estimates in problem settings where large weights arise with high probability. We study its sampling variance as a function of the training data distribution and introduce a control variate to increase its robustness to problematically large weights.
BibTeX
@misc{kouw2017reducing,
  author = {Wouter M. Kouw and Marco Loog},
  title = {{On reducing sampling variance in covariate shift using control variates}},
  year = {2017},
  eprint = {1710.06514},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1710.06514},
}

Loading related content...

Figure