Treatment Choice with Nonlinear Regret

Toru Kitagawa, Sokbae Lee, Chen QiuarXiv 2022 · DOI · arXiv

In the world of policy-making and clinical trials, the goal is simple: use data to choose the treatment that maximizes social welfare. Traditionally, researchers have focused on minimizing "expected regret"—the average welfare lost by not knowing the "true" best treatment in advance. However, a new paper by Toru Kitagawa, Sokbae Lee, and Chen Qiu suggests that this traditional focus might be making us too aggressive and sensitive to random noise.

TL;DR

This paper moves beyond minimizing average regret to minimizing a nonlinear transformation of regret (specifically "mean square regret"). This shift justifies using fractional treatment rules—treating only a portion of the population when evidence is shaky—providing a more stable, "regret-averse" framework for decision-making.

What Problem Does This Paper Address?

Standard statistical decision theory often leads to "singleton rules": based on your sample, you either treat everyone or no one. For example, if a trial shows a tiny positive effect, the "Empirical Success" rule dictates treating the entire population.

The problem? This approach is incredibly sensitive to sampling uncertainty. A single data point swinging from a "success" to a "failure" could flip the decision from 100% treatment to 0%. This "all-or-nothing" aggressiveness ignores the variance of the outcome. A decision-maker might be "regret-averse," meaning they are more worried about a massive welfare loss in a "bad state of the world" than they are excited about a marginal gain in a "good" one.

Methods

The authors propose a novel framework grounded in statistical decision theory that optimizes a nonlinear transformation of welfare regret.

  • Nonlinear Regret Risk: Instead of minimizing $E[Regret]$, they minimize $E[g(Regret)]$.
  • Mean Square Regret: They focus heavily on the case where $g(r) = r^2$. Mathematically, this is equivalent to minimizing the sum of the squared expected regret and the variance of the regret. This explicitly penalizes rules that lead to highly volatile welfare outcomes.
  • Fractional Rules: Unlike traditional models where fractional rules (treating, say, 60% of people) are often seen as "suboptimal," the authors prove that under nonlinear regret, singleton rules are often inadmissible (meaning there's always a better fractional rule) or incomplete.
  • Asymptotic Theory: They extend their finite-sample Gaussian results to "limit experiments," allowing the framework to be applied to large-sample parametric models using standard t-statistics.

Datasets

The paper is primarily theoretical and methodological. To evaluate and illustrate their findings, the authors use:

  • Simulated Experiments: Specifically, a Gaussian experiment ($N(1, 1)$) with 10,000 simulations to compare the empirical distributions of regret and welfare between their proposed rule and standard rules.
  • Normal Regression Models: They apply the theory to a standard regression specification frequently used by practitioners to demonstrate how to calculate optimal treatment fractions in real-world scenarios.

Key Insights and Ideas

  • The Logistic Connection: One of the most elegant findings is that the minimax optimal rule for mean square regret follows a logistic structure. The fraction of the population treated is a function of the t-statistic: $$\frac{\exp(2 \cdot 1.23 \cdot \text{t-statistic})}{\exp(2 \cdot 1.23 \cdot \text{t-statistic}) + 1}$$
  • Strength of Evidence: The treatment fraction $\hat{\delta}$ can be interpreted as a summary statistic for the "strength of evidence." If the rule suggests treating 54% of the population, it signals that the evidence favoring treatment is not yet entirely strong.
  • Regret Aversion: The model provides a microeconomic foundation for "regret aversion," distinct from standard ambiguity aversion. It captures the intuition that we should be cautious when data is noisy.
  • Thompson Sampling: The authors show that their minimax rule coincides with "posterior probability-matching" (similar to the Thompson sampling algorithm used in multi-armed bandits), providing a new justification for this popular algorithm in static settings.

Key Results

  • Risk Reduction: In a simulated $N(1, 1)$ experiment, the standard Empirical Success (ES) rule had a 16% probability of incurring the maximum possible regret. The authors' proposed minimax rule reduced this "nightmare scenario" probability to just 1.4%.
  • Sample Size Efficiency: The authors demonstrate that if a planner insists on using a singleton rule but wants the stability (low variance) of the nonlinear regret framework, they would need significantly more data. To guarantee the same worst-case mean square regret as the authors' rule, the standard ES rule requires 40% more observations, while a standard Hypothesis Testing rule might require 11 times more data.
  • Closed-Form Solutions: The paper provides ready-to-use formulas for both Bayes and minimax optimal rules that are "tuning-parameter free," making them easy for practitioners to implement.

Limitations and Future Work

  • Covariates: While the authors note that incorporating discrete covariates is straightforward (by applying the rule to each subpopulation), the paper does not deeply explore high-dimensional or continuous covariate settings.
  • Policy Implementation: In practice, treating a "fraction" of a population can be ethically or logistically complex. The authors suggest implementing this via randomization devices or interpreting the fraction as a degree of confidence, but the political hurdle of "partial treatment" remains.
  • Functional Form: The focus is largely on the quadratic ($r^2$) case. While they provide general theorems for other $g(\cdot)$ functions, closed-form solutions for more exotic transformations might be less tractable.

Full Paper Citation: Toru Kitagawa, Sokbae Lee, Chen Qiu. "Treatment Choice with Nonlinear Regret." arXiv:2205.08586v6 [econ.EM], Oct 2024.

Summary updated

Abstract

The literature focuses on the mean of welfare regret, which can lead to undesirable treatment choice due to sensitivity to sampling uncertainty. We propose to minimize the mean of a nonlinear transformation of regret and show that singleton rules are not essentially complete for nonlinear regret. Focusing on mean square regret, we derive closed-form fractions for finite-sample Bayes and minimax optimal rules. Our approach is grounded in decision theory and extends to limit experiments. The treatment fractions can be viewed as the strength of evidence favoring treatment. We apply our framework to a normal regression model and sample size calculation.
BibTeX
@misc{kitagawa2022treatment,
  author = {Toru Kitagawa and Sokbae Lee and Chen Qiu},
  title = {{Treatment Choice with Nonlinear Regret}},
  year = {2022},
  eprint = {2205.08586},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2205.08586},
}
Figure