Is Your Model Predicting the Past?

Michael P. Kim, Moritz Hardt

TL;DR

This paper introduces "backward baselines," a novel method to distinguish between machine learning models that predict future outcomes based on individual behavior versus those that simply reproduce historical patterns. The authors demonstrate that many models rely heavily on "backward prediction," which has significant implications for fairness and the utility of interventions.

What Problem Does This Paper Address?

Machine learning models are increasingly used for consequential decision-making, with proponents arguing they can predict future individual outcomes. However, critics caution that these models may simply be reflecting and perpetuating historical biases and social stratification, rather than truly predicting the future. This raises a fundamental question: is a model predicting an individual's future agency, or is it reciting patterns from the past that predate the individual? The authors aim to provide a rigorous framework to disentangle these two prediction pathways.

Methods

The core contribution is the development of "backward baselines," a family of simple yet effective statistical tests. These baselines quantify how much of a predictor's strength can be attributed to background variables (factors determined before the individual, like demographics) versus individual factors (variables the individual can influence). The theory establishes equivalences between these baselines and familiar statistical concepts. The authors also introduce formal definitions of "backward prediction" and "forward prediction" based on conditional independence. They propose methods to estimate these baselines even when only black-box access to the model and background variables are available, without requiring true outcome labels.

Datasets

The authors evaluate their framework on several longitudinal panel surveys and a notorious dataset:

  • Medical Expenditure Panel Survey (MEPS): Used to predict year-round medical expenditure.
  • Survey of Income and Program Participation (SIPP): Used to predict poverty.
  • ProPublica COMPAS Recidivism Scores: Used to audit the COMPAS recidivism prediction model.

Key Insights and Ideas

  • Backward prediction is prevalent: Many models, even those aiming to predict future outcomes, rely heavily on background variables that predate individual actions. This means they are essentially "reciting the past."
  • Implications for intervention: If a model primarily uses backward prediction, it offers little advantage for individual-level interventions. Interventions based on such predictions are akin to stereotyping and randomizing within demographic groups.
  • Auditing without labels: The backward baselines can be evaluated even without access to the true outcome variable, making them valuable for auditing proprietary models.
  • Forward prediction is orthogonal to backward prediction: The framework suggests that forward and backward prediction are distinct and can be analyzed separately, offering a more nuanced understanding of model behavior.

Key Results

The empirical evaluations across the MEPS, SIPP, and COMPAS datasets consistently show that backward prediction plays a significant role in the predictive power of the models. For instance, in the MEPS dataset, predictors trained only on background variables achieved nearly the same predictive performance as classifiers trained on all features. Similarly, for the COMPAS scores, a single feature (prior convictions) appeared to account for all of its predictive power, highlighting the dominance of backward prediction.

Limitations and Future Work

The authors acknowledge that choosing the "right" background variables (W) is not purely a technical question but depends on the context and scope of the prediction task. Time is a guiding principle but imperfect. Future work could explore more sophisticated methods for selecting W and further investigate the interplay between backward baselines and fairness notions like demographic parity. The paper also suggests that backward baselines can be a valuable addition to the standard "report card" for supervised learning models, providing crucial interpretability.

Summary updated

BibTeX
@inproceedings{kim2023is,
  author = {Michael P. Kim and Moritz Hardt},
  title = {{Is Your Model Predicting the Past?}},
  booktitle = {EAAMO},
  year = {2023},
  doi = {10.1145/3617694.3623225},
}
Figure