Introduction
Inverse Decision Modeling (IDM) is a novel framework designed to move beyond simply mimicking behavior to understanding the "why" behind it. By explicitly modeling bounded rationality, IDM allows researchers to quantify the internal biases and information-processing constraints of a decision-maker—such as their level of optimism, flexibility, and adaptivity. This makes it a powerful tool for auditing human decision-making in high-stakes fields like healthcare.
The Gap Between "Perfect" and "Observed" Behavior
In artificial intelligence, behavior is typically modeled in two ways: normative analysis (how a perfect agent should act, often via Reinforcement Learning) or descriptive analysis (how an agent does act, often via Imitation Learning).
However, there is a massive gap in the middle. Real-world agents—like doctors or investors—are rarely "perfect." They are boundedly rational, meaning their decisions are limited by biological, psychological, and computational constraints. Standard Imitation Learning (IL) merely copies these mistakes without understanding them, while Inverse Reinforcement Learning (IRL) usually assumes the agent is trying to be optimal, leading to "misinterpreting" biases as part of the agent's core goals. This paper asks a critical question: How can we intelligibly characterize imperfect behavior?
The Inverse Decision Modeling (IDM) Framework
To address this, the authors propose Inverse Decision Modeling (IDM), a unifying framework that formalizes two distinct directions of behavior modeling.
- The Forward Problem: A "planner" takes parameters representing goals and biases and produces a behavior. Crucially, this includes a recognition policy (how the agent updates internal beliefs from observations) and a decision policy (how they choose actions).
- The Inverse Problem: Given observed behavior, IDM "projects" that behavior onto a parameterized space to infer the agent's internal state.
The core of their implementation is Inverse Bounded Rational Control (IBRC). They use information-theoretic penalties (KL-divergence) to represent three types of "boundedness":
- Decision Complexity ($\alpha^{-1}$): How much effort the agent puts into choosing an action versus following a baseline (Flexibility).
- Specification Complexity ($\beta^{-1}$): The agent's "optimism" or "pessimism" regarding their internal model of the world.
- Recognition Complexity ($\eta^{-1}$): The agent's "adaptivity" or how much they are surprised by new evidence.
Key Insights and Results
The paper evaluates the framework using a synthetic medical diagnosis environment (DIAG) and real-world data from the Alzheimer’s Disease Neuroimaging Initiative (ADNI).
A major insight of IDM is the decoupling of beliefs from actions. Suboptimality comes from two distinct sources: an agent might have the right goal but lack the "processing power" to act on it, or they might have a fundamentally biased "mental model" of how the world works. IDM can tell the difference.
- Superior Expressivity: In the DIAG simulation, IDM successfully distinguished between two agents who both stopped testing early. Standard IRL saw them as identical. IDM, however, correctly identified that one agent was overreacting to evidence (high optimism), while the other simply had a lower threshold for stopping (low adaptivity).
- Real-world Auditing: Applying IDM to the ADNI Alzheimer's dataset revealed that clinicians are significantly less optimistic (more cautious) when diagnosing patients with the ApoE4 genetic risk factor compared to other groups.
- Parameter Recovery: Using Bayesian inference, the authors showed that IDM can accurately recover the "true" flexibility, optimism, and adaptivity parameters from generated behavior trajectories, proving the model is identifiable.
Limitations and Future Work
While powerful, IDM faces classic identifiability challenges: in a single environment, it is often impossible to distinguish between an agent's "utility function" (what they want) and "reward shaping" (how they get it). Furthermore, the framework's success depends on choosing the right structured parameterization. Future work will explore how well these interpretable models scale to high-dimensional state spaces where simple state-based reasoning might break down.
Summary updated