On Human-Aligned Risk Minimization

Liu Leqi, Adarsh Prasad, Pradeep RavikumarNeurIPS 2019

Traditional machine learning optimizes for the "average" case, often ignoring rare but catastrophic failures. This paper proposes Empirical Human Risk Minimization (EHRM), a training objective inspired by how humans actually perceive risk through the lens of Cumulative Prospect Theory. By adopting this "human-like" approach, the authors demonstrate that models naturally improve in fairness and performance on minority subgroups.

The Problem with Average Risk

Most machine learning models are trained using Empirical Risk Minimization (ERM), which aims to minimize the expected (average) loss. While this works well for general accuracy, it has a major flaw: it treats all errors of the same magnitude equally, regardless of how often they occur or which group they affect.

In the real world, humans do not evaluate risk this way. According to behavioral economics, humans are "risk-averse" regarding large losses and tend to overweight the importance of rare, extreme events. By focusing only on the average, standard ML models often achieve high overall accuracy by "sacrificing" minority subgroups or failing spectacularly in rare but critical scenarios. The authors ask a compelling question: What if we trained models to minimize risk the way humans actually perceive it?

Methods: The Inverse S-Shaped Weighting Function

To bridge this gap, the authors draw from Cumulative Prospect Theory (CPT) to define a new objective function. The core innovation is the use of an inverse S-shaped probability weighting function.

In standard ERM, every data point's loss is weighted by its uniform probability ($1/n$). In EHRM, the authors apply a non-linear weight to the cumulative distribution of the loss. This function does two key things:

  1. Inflates small probabilities: Rare events (like high losses on a minority subgroup) are given more weight.
  2. Deflates large probabilities: "Average" events that happen frequently are given relatively less weight.
The inverse S-shaped probability weighting function ($w_{POLY}$) and its derivative, illustrating how EHRM inflates the weights of rare events.

Mathematically, they introduce a polynomial weighting function $w_{POLY}$ controlled by two parameters: $a$ (the fixed point) and $b$ (the curvature/sensitivity). They then develop an iterative optimization rule—a modified version of Gradient Descent—that re-weights gradients based on the "rank" of the loss for each instance during training.

Key Insights and Ideas

  • Fairness as a Side Effect of Risk Aversion: The most surprising insight is that explicit "fairness constraints" are not strictly necessary to make a model fair. By simply penalizing extreme losses (which often occur in minority groups that the model hasn't learned well), the model naturally balances its performance across all subgroups.
  • Sensitivity to Skewness: Standard ERM is "blind" to the skewness of a loss distribution; it only cares about the mean. EHRM, however, can distinguish between two models with the same average error but different "tail" behaviors, preferring the model that avoids catastrophic outliers.
  • The "Surprisal" Connection: The authors link their weighting function to the information-theoretic concept of "surprisal." Effectively, the model learns to pay more attention to data points that provide the most "surprising" (rare and high-loss) information.

Empirical Results

The authors evaluated their approach across three distinct settings, proving its efficacy on both synthetic and real-world data:

  1. Synthetic Regression: In a controlled environment where a "minority" group (5% of the data) has a different underlying distribution than the majority, EHRM significantly reduced the risk for the minority subgroup compared to Ordinary Least Squares (OLS).
Comparison of Majority and Minority Risk across OLS, CVaR, and HRM, demonstrating the trade-off between overall performance and minority group fairness.

As seen above, EHRM effectively trades a tiny bit of majority-group performance for a much more equitable outcome for the minority.

  1. Recidivism Prediction (COMPAS): On a real-world dataset used to predict criminal re-offending—frequently studied for its inherent biases—EHRM produced much more consistent misclassification rates across different demographic groups (e.g., age, race, gender) compared to standard ERM.
  2. Gender Classification (UTKFace): In tests on a large-scale facial dataset, EHRM outperformed standard ERM and even specialized "reweighting" fairness algorithms across several metrics, including Disparate Impact and Equal Opportunity Difference. For example, EHRM achieved a Disparate Impact score of 0.8475 (closer to 1.0 is better) compared to ERM's 0.8368, while maintaining competitive accuracy.

Limitations and Future Work

While promising, EHRM introduces new challenges. The optimization method used is a heuristic, and deriving a provably optimal algorithm for minimizing human-aligned risk remains an open theoretical challenge. Additionally, the performance of EHRM depends heavily on the choice of hyperparameters $a$ and $b$. The "correct" human alignment might vary depending on the specific application (e.g., medical diagnosis vs. ad targeting). Finally, calculating the empirical CDF (ranking the losses) at each step adds computational complexity compared to standard SGD, which may complicate scaling to massive datasets.

Summary updated

BibTeX
@inproceedings{leqi2019human,
  author = {Liu Leqi and Adarsh Prasad and Pradeep Ravikumar},
  title = {{On Human-Aligned Risk Minimization}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2019},
}
Figure