From Predictive Algorithms to Automatic Generation of Anomalies

Sendhil Mullainathan, Ashesh Rambachan

Abstract

Machine learning algorithms can find predictive signals that researchers fail to notice; yet they are notoriously hard-to-interpret. How can we extract theoretical insights from these black boxes? History provides a clue. Facing a similar problem—how to extract theoretical insights from their intuitions—researchers often turned to "anomalies:" constructed examples that highlight flaws in an existing theory and spur the development of new ones. Canonical examples include the Allais paradox and the Kahneman-Tversky choice experiments for expected utility theory. We suggest anomalies can extract theoretical insights from black box predictive algorithms. We develop procedures to automatically generate anomalies for an existing theory when given a predictive algorithm. We cast anomaly generation as an adversarial game between a theory and a falsifier, the solutions to which are anomalies: instances where the black box algorithm predicts—were we to collect data—we would likely observe violations of the theory. As an illustration, we generate anomalies for expected utility theory using a large, publicly available dataset on real lottery choices. Based on an estimated neural network that predicts lottery choices, our procedures recover known anomalies and discover new ones for expected utility theory. In incentivized experiments, subjects violate expected utility theory on these algorithmically generated anomalies; moreover, the violation rates are similar to observed rates for the Allais paradox and Common ratio effect.

Introduction: The Challenge of Black Box Interpretability

In fields like economics and behavioral science, machine learning (ML) models often significantly outperform traditional theories in predictive accuracy. However, these ML models are frequently "black boxes"—they provide accurate predictions of what will happen without explaining the underlying mechanisms or "why" it happens. This creates a gap between predictive power and scientific understanding.

Historically, scientific progress has often been driven by the discovery of anomalies: hand-crafted examples that highlight where a theory's logic breaks down. For instance, the Allais paradox famously challenged Expected Utility Theory (EUT) by demonstrating systematic human departures from its axioms. While finding these anomalies has traditionally required immense human intuition, this research explores whether the predictive power of ML can be harnessed to automatically identify these theoretical "cracks."

Methodology: Automating the Search for Cracks

The authors propose a formal econometric framework that treats a theory as a set of "allowable functions." They introduce two primary procedures to identify anomalies:

  1. The Adversarial Algorithm: This approach casts anomaly generation as a max-min adversarial game. A "falsifier" (the ML model) proposes scenarios, while the "theory" attempts to find its best-fitting function to explain them. An anomaly is identified when the falsifier finds a scenario where even the theory's best attempt results in high prediction error. This is solved using Gradient Descent Ascent (GDA).
  2. Example Morphing: This procedure identifies "representational anomalies" by taking an existing scenario and using projected gradient descent to find a nearby version where the theory predicts the outcome should remain constant, but the ML model predicts a significant change.

Experimental Setup and Datasets

The framework was validated using three distinct data sources:

  • Simulated Data: Generated from Prospect Theory to test if the algorithms could rediscover known anomalies.
  • Choices13K: A massive dataset containing over 1 million incentivized decisions from 14,711 workers on nearly 10,000 lottery menus.
  • New Experimental Data: Incentivized experiments conducted on Prolific with approximately 1,000 participants to test the newly discovered, algorithmically generated anomalies.

Results: Rediscovery and New Discoveries

Using only a neural network's predictions, the procedures successfully reconstructed classic anomalies like the Allais paradox and the Certainty effect.

The Allais Paradox: A classic anomaly where human choices violate Expected Utility Theory.

Beyond rediscovery, the algorithms identified entirely new patterns of behavior, such as the "Dominated Consequence Effect" and the "Strict Dominance Effect." These represent systematic human departures from EUT that had not been formally categorized previously. In new human experiments, the violation rates for these algorithmically generated anomalies ranged from 7.2% to 14.5%. Crucially, these rates are comparable to the violation rates of the most celebrated paradoxes in behavioral economics, such as the Allais paradox (~16%) and the Certainty effect (~8.3%).

Discussion and Future Directions

This research demonstrates that anomalies can serve as a bridge between black box models and scientific discovery. Instead of attempting to "open" the neural network directly, the authors use its predictions to generate concrete, interpretable examples that humans can analyze. This turns ML from a competitor to theory into a tool for building more robust ones.

While promising, the method faces optimization challenges due to the non-convex nature of the adversarial game, which may lead to missing some anomalies. Furthermore, while the framework is general, applying it to more complex domains like Nash equilibrium or asset pricing will require further refinement. Ultimately, the transition from finding anomalies to constructing superior new theories still requires the synthesis and mathematical creativity of human researchers.

Summary updated

Abstract

Machine learning algorithms can find predictive signals that researchers fail to notice; yet they are notoriously hard-to-interpret.How can we extract theoretical insights from these black boxes?History provides a clue.Facing a similar problem -how to extract theoretical insights from their intuitions -researchers often turned to "anomalies:" constructed examples that highlight flaws in an existing theory and spur the development of new ones.Canonical examples include the Allais paradox and the Kahneman-Tversky choice experiments for expected utility theory.We suggest anomalies can extract theoretical insights from black box predictive algorithms.We develop procedures to automatically generate anomalies for an existing theory when given a predictive algorithm.We cast anomaly generation as an adversarial game between a theory and a falsifier, the solutions to which are anomalies: instances where the black box algorithm predicts -were we to collect data -we would likely observe violations of the theory.As an illustration, we generate anomalies for expected utility theory using a large, publicly available dataset on real lottery choices.Based on an estimated neural network that predicts lottery choices, our procedures recover known anomalies and discover new ones for expected utility theory.In incentivized experiments, subjects violate expected utility theory on these algorithmically generated anomalies; moreover, the violation rates are similar to observed rates for the Allais paradox and Common ratio effect.
BibTeX
@misc{mullainathan2024from,
  author = {Sendhil Mullainathan and Ashesh Rambachan},
  title = {{From Predictive Algorithms to Automatic Generation of Anomalies}},
  year = {2024},
  eprint = {2404.10111},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2404.10111},
}
Figure