In classical machine learning, we treat data as a static snapshot of the world. We assume that the distribution of features and labels is fixed, and our only job is to find the pattern. But in high-stakes social domains—like credit scoring, college admissions, or insurance—the "data points" are people. These people are strategic: they observe the model's decision rules and adapt their behavior to achieve a better outcome.
This shift from static data to reactive agents transforms machine learning from a simple optimization problem into a game-theoretic challenge.
The Feedback Loop of Performative Prediction
When a model’s deployment changes the very distribution it aims to predict, we enter the realm of performative prediction. As Juan Carlos Perdomo, Tijana Zrnic, and Moritz Hardt argue in Performative Prediction, a classifier is not just a passive observer; it is an intervention.
Imagine a bank that uses "number of open credit lines" to predict default risk. Once this rule is public, applicants might close accounts specifically to boost their scores. The model is no longer predicting a pre-existing state; it is setting the incentives that define the future data it will encounter. This creates a feedback loop where the relationship between features and outcomes is constantly shifting. The goal in this setting is to find a performatively stable point—a model that remains optimal even after the population has adapted to it.
Modeling the Interaction as a Stackelberg Game
To build intuition for how to solve this, researchers often frame the problem as a Stackelberg game. The model designer (the leader) moves first by deploying a classifier. The agents (the followers) move second, modifying their features to maximize their utility (e.g., getting a loan) while minimizing the "cost" of that modification.
In Strategic Classification from Revealed Preferences, Aaron Roth, Zhiwei Steven Wu, and colleagues show that we can learn optimal classifiers even if we don't know exactly how agents will react. By observing "revealed preferences"—how people change their data in response to different models—we can iteratively update the classifier to minimize "Stackelberg regret." This approach acknowledges that the learner only sees the manipulated data, not the original, uncorrupted features.
Gaming versus Constructive Adaptation
A central tension in this field is the difference between "gaming" and "improvement." * Gaming occurs when an agent modifies a feature that is correlated with the outcome but has no causal link to it (e.g., changing the font on a resume). * Improvement (or recourse) occurs when the agent modifies a feature that actually changes the ground truth (e.g., taking a training course to become more qualified).
Yatong Chen and others argue in Linear Classifiers that Encourage Constructive Adaptation that we should design classifiers that incentivize the latter. If a model relies on non-causal features, it is vulnerable to manipulation that degrades its accuracy. However, as Guy Horowitz and Nir Rosenfeld explore in Causal Strategic Classification: A Tale of Two Shifts, if the features we incentivize are causally linked to the outcome, the strategic behavior of agents can actually lead to a "better" population over time.
Why Fairness and Robustness Break
Strategic behavior breaks many of the guarantees we take for granted in standard ML. For instance, Andrew Estornell and Sanmay Das demonstrate a "fairness reversal" in Unfairness Despite Awareness. They show that a model designed to be fair on static data can become more unfair once deployed, because different social groups may have different costs for modifying their features. If one group finds it harder to "game" the system, a seemingly fair model will disproportionately exclude them.
Similarly, the statistical limits of learning change. Ravi Sundaram and Haifeng Xu introduced the Strategic VC-dimension (SVC) in PAC-Learning for Strategic Classification to characterize the complexity of learning when agents can move data points across decision boundaries. This framework helps us understand when robust classification is mathematically possible and when the ability of agents to manipulate features makes the learning problem intractable.
From Prediction to Incentive Design
The core insight of this research direction, as summarized by Chara Podimata in Incentive-Aware Machine Learning, is that in social systems, prediction is incentive design.
When we deploy a model, we are not just estimating a value; we are publishing a set of instructions for how to succeed in a system. By moving beyond the assumption of static data and embracing the strategic nature of human subjects, we can build models that are not only more robust to "gaming" but also more effective at encouraging genuine self-improvement.