Introduction
Observing humans making tough calls under uncertainty reveals their true preferences far better than observing "obvious" decisions. This paper provides the first statistical analysis of Inverse Decision Theory (IDT), proving that uncertainty is not a hurdle but a requirement for precisely identifying the internal trade-offs a decision-maker is willing to make.
The standard way to learn human preferences from behavior is Inverse Reinforcement Learning (IRL). However, IRL is notoriously "underspecified"—many different sets of preferences can explain the same observed behavior. This is especially true when decisions are "clear" or "obvious." If everyone chooses the same path because it's clearly better, we learn nothing about their internal values or the specific trade-offs they would make in a closer call.
Furthermore, most existing methods assume humans are perfectly optimal, which is rarely true. When humans are suboptimal, it becomes even harder to distinguish whether their behavior stems from a specific preference or just a mistake. This paper tackles these challenges by studying decisions made under uncertainty (partial observability), arguing that these "fuzzy" scenarios are actually the most informative for learning what a person truly values.
Methods: Inverse Decision Theory
To address these challenges, the authors utilize the framework of Inverse Decision Theory (IDT). Unlike sequential IRL, IDT focuses on non-sequential binary decisions. Key components of their approach include:
- Loss Function Parameterization: They simplify the human's internal loss function into a single parameter $c$, representing the cost of a false positive relative to a false negative.
- Threshold Identification: They show that an optimal decision-maker essentially uses a threshold on the posterior probability of a "correct" outcome. Learning their preferences is equivalent to identifying this threshold.
- Suboptimality Modeling: They extend the analysis to two types of suboptimal humans:
- Known Suboptimal: The human chooses the best rule within a restricted (and known) set of possible rules.
- Unknown Suboptimal: The human uses a rule from an unknown family of hypothesis classes (e.g., they might be ignoring certain data features, but we don't know which ones).
- Statistical Analysis: They provide the first formal sample complexity bounds—calculating exactly how many observations are needed to learn the preference parameter $c$ to a desired precision $\epsilon$.
While theoretical, the paper grounds its math in real-world scenarios, such as an ER doctor deciding whether to treat a patient based on uncertain test results, or a judge deciding on a conviction based on circumstantial evidence.
Key Insights and Ideas
- Uncertainty is a Feature, Not a Bug: In a "clear" decision, the threshold for a decision could be anywhere, making the loss function unidentifiable. In an "uncertain" decision, the human is forced to reveal their threshold, making their preferences visible.
- The "Density" of Decisions Matters: The ease of learning preferences depends on the probability density ($p_c$) of observations near the decision-maker's internal threshold. If a human rarely faces "borderline" cases, it takes much longer to learn their preferences.
- Suboptimality is Manageable: Surprisingly, we can still identify exact preferences even if a human is suboptimal, provided their suboptimality follows a "monotone" structure (e.g., they consistently ignore certain features).
- Fairness Testing: The authors show that IDT can be used to test for "group calibration" or "sufficiency" in human decision-making—essentially checking if a decision-maker is applying the same internal "cost" to different demographic groups.
Key Results
- Optimal Sample Complexity: For an optimal decision-maker, the number of samples $m$ needed to estimate the preference $c$ within error $\epsilon$ is $O(1/(p_c \epsilon))$.
- Impossibility Theorem: The authors formally prove that if there is no uncertainty in the decision problem, it is impossible to identify the true loss parameter $c$ to any non-trivial precision.
- Suboptimal Complexity: For humans using an unknown hypothesis class, the complexity is higher, growing with $1/\epsilon^2$ and the VC-dimension of the hypothesis classes, but it remains learnable.
- Fairness Link: They prove that if a decision-maker uses the same loss parameter $c$ across different groups, their decisions will satisfy the group calibration fairness criterion.
Limitations and Future Work
The analysis assumes the observer knows the true underlying distribution $\mathcal{D}$ of the data, which is often not the case in practice. Furthermore, the work assumes humans act to minimize expected loss, not yet accounting for more complex behavioral models like Prospect Theory, where humans perceive risks and gains non-linearly. Finally, while IDT provides a strong foundation for non-sequential choices, many real-world preferences are revealed through sequences of actions, which this paper leaves for future extension.
Summary updated