Model Transferability With Responsive Decision Subjects

Yang Liu, Yatong Chen, Zeyu Tang, Kun Zhang

Abstract

Given an algorithmic predictor that is accurate on some source population consisting of strategic human decision subjects, will it remain accurate if the population responds to it? In our setting, an agent or a user corresponds to a sample $(X,Y)$ drawn from a distribution $\mathcal{D}$ and will face a model $h$ and its classification result $h(X)$. Agents can modify $X$ to adapt to $h$, which will incur a distribution shift on $(X,Y)$. Our formulation is motivated by applications where the deployed machine learning models are subjected to human agents, and will ultimately face responsive and interactive data distributions. We formalize the discussions of the transferability of a model by studying how the performance of the model trained on the available source distribution (data) would translate to the performance on its induced domain. We provide both upper bounds for the performance gap due to the induced domain shift, as well as lower bounds for the trade-offs that a classifier has to suffer on either the source training distribution or the induced target distribution. We provide further instantiated analysis for two popular domain adaptation settings, including covariate shift and target shift.

Introduction: When Data Fights Back

In standard machine learning, we typically assume that data is a static "snapshot" of the world. However, in high-stakes social contexts—such as loan approvals, insurance pricing, or college admissions—the act of deploying a model changes the very environment it aims to predict. This phenomenon is a manifestation of Goodhart’s Law: "When a measure becomes a target, it ceases to be a good measure."

When a bank uses specific features like "debt" or "savings" for credit decisions, applicants may strategically modify their behavior to improve their chances of approval. This creates an induced distribution shift. Unlike traditional domain adaptation, where shifts occur due to external environmental changes, here the shift is a direct response to the model itself.

An example of strategic feature adaptation where a subject modifies debt and savings to improve their outcome based on model weights.

The central challenge is transferability: can we predict how a model trained on historical, static data will perform once it faces a responsive population? This paper provides a theoretical framework to quantify these performance gaps before a model is even deployed.

Formalizing Induced Risk

The researchers formalize this problem through the lens of Induced Risk—the error a model $h$ incurs on the distribution $D(h)$ that it induces. This framework bridges the gap between Strategic Classification (rational individual adaptation) and Performative Prediction (population-level shifts).

To model these interactions, the authors utilize Directed Acyclic Graphs (DAGs) to represent causal relationships between features ($X$) and labels ($Y$). As shown in Figure 1, the deployment of a model $h(X)$ acts as a causal intervention that transforms the source distribution into a target distribution $(X', Y')$.

Causal DAGs illustrating how a model $h(X)$ induces a distribution shift from the source $(X, Y)$ to the target $(X', Y')$, covering both covariate and target shift scenarios.

Key Technical Contributions

  • Performance Bounds: Using tools like $\mathcal{H}$-divergence and Total Variation distance, the paper derives upper bounds on the "transfer error." For strategic classification, they prove that this error is bounded by the "manipulation budget" $B$ (the effort agents can exert to change features).
  • Learning Trade-offs: The authors prove that a classifier often cannot be simultaneously optimal on both the original and the responding populations, establishing a fundamental trade-off.
  • Specialized Shift Analysis: The framework is applied to Covariate Shift (where features $X$ change but labels $Y$ remain stable) and Target Shift (where labels $Y$ shift, modeled using Replicator Dynamics from evolutionary game theory).

Experimental Validation

The theoretical framework was validated using two distinct data sources:

  1. Synthetic DAG-based Data: Simulations allowed for precise control over how features respond to model thresholds, confirming that theoretical bounds effectively "sandwich" observed performance gaps.
  2. FICO Credit Score Dataset: Using over 300,000 real-world records, the authors simulated a "repeated resource allocation" scenario where credit scores evolve based on model-driven approvals or denials.

Key Insights

A critical finding (Theorem 4.2) highlights that model suboptimality depends heavily on the variance of adaptation. If a model induces wildly different levels of response across a population, its eventual accuracy becomes significantly harder to predict. Furthermore, the study demonstrates that standard L1 or L2 regularization techniques are insufficient for robustness; models must explicitly account for the inducing-risk mechanism.

Conclusion

This work moves machine learning from a passive observation task to an interactive system design task. While the derived bounds provide a crucial safety check for deploying models in sensitive domains, they currently rely on an assumed "response model" of human behavior. Future research must address the messiness of real-world human rationality and the computational challenges of optimizing models for non-convex induced distributions.

Summary updated

Abstract

Given an algorithmic predictor that is accurate on some source population consisting of strategic human decision subjects, will it remain accurate if the population respond to it? In our setting, an agent or a user corresponds to a sample $(X,Y)$ drawn from a distribution $\cal{D}$ and will face a model $h$ and its classification result $h(X)$. Agents can modify $X$ to adapt to $h$, which will incur a distribution shift on $(X,Y)$. Our formulation is motivated by applications where the deployed machine learning models are subjected to human agents, and will ultimately face responsive and interactive data distributions. We formalize the discussions of the transferability of a model by studying how the performance of the model trained on the available source distribution (data) would translate to the performance on its induced domain. We provide both upper bounds for the performance gap due to the induced domain shift, as well as lower bounds for the trade-offs that a classifier has to suffer on either the source training distribution or the induced target distribution. We provide further instantiated analysis for two popular domain adaptation settings, including covariate shift and target shift.
BibTeX
@inproceedings{liu2023model,
  author = {Yang Liu and Yatong Chen and Zeyu Tang and Kun Zhang},
  title = {{Model Transferability With Responsive Decision Subjects}},
  booktitle = {International Conference on Machine Learning},
  year = {2023},
  url = {https://openreview.net/forum?id=IGfmSM7siu},
}
Figure