Linear decision rule as aspiration for simple decision heuristics

Özgür SimsekNeurIPS 2013 · OpenReview

Overview

Simple decision heuristics often perform surprisingly well compared to complex linear models. This paper demonstrates that this success isn't a fluke; rather, it's because natural environments are rich in structures like "dominance" and "noncompensation." These structures allow simple rules to mimic the decisions of a full linear model using only a fraction of the information and computation.

The Comparison Problem

In decision science, the "comparison problem" asks: given two objects with several known attributes, which one ranks higher on an unobserved criterion? For example, which of two houses will have a higher resale value in ten years?

Standard statistical practice suggests using a linear decision rule—weighting and summing all available attributes to make a prediction. However, humans and animals often use simple heuristics, such as "Take-the-Best" (deciding based solely on the single most important attribute).

The author explores why these simple rules work so well. Are they just "good enough" approximations, or does the structure of real-world data render the full linear model's complexity unnecessary? The paper investigates three specific environmental structures: dominance, cumulative dominance, and noncompensatoriness.

Methodology and Datasets

The author formalizes the conditions under which a simple rule yields the same decision as a linear model:

  1. Dominance: Object A is better than B on at least one attribute and no worse on any others.
  2. Cumulative Dominance: A more relaxed version of dominance that considers the relative ordering of weights.
  3. Noncompensatoriness: A condition where the "weight" of the most important attribute is greater than the sum of all other weights combined, meaning no combination of lesser attributes can "compensate" for a disadvantage on the primary one.

A Novel Probabilistic Approach: Instead of requiring 100% certainty for dominance, the author introduces a normal approximation to estimate the probability that one object dominates another. This allows for "approximate dominance," where a decision can be made with high confidence (e.g., 99%) without knowing the exact weights of the linear model.

To test these ideas, the author trained "base" linear models using Elastic Net regularization on 51 natural datasets spanning biology, business, computer science, ecology, economics, and sports. The datasets varied in size from 12 to 601 objects, resulting in up to 180,300 distinct paired comparisons. Analyses were performed on both the original numeric data and "binarized" versions.

Key Insights

  • Cumulative dominance is the "sweet spot": While simple dominance is relatively rare in numeric data, cumulative dominance is highly prevalent. It allows simple rules to reach near-perfect parity with linear models while only needing to know the rank order of attribute importance, not the exact values.
  • Noncompensatory weights vs. Noncompensatory decisions: Strictly noncompensatory weights (where the first cue is mathematically unbeatable) are somewhat rare, found in only 17% of models.
Distribution of noncompensatory weights

However, actual noncompensation—where the first discriminating cue happens to match the final linear decision—is overwhelmingly common (median of 96% in binary data).

  • Structure, not just "frugality": The success of heuristics isn't just about being "fast and frugal"; it's about exploiting the fact that in the real world, cues are often redundant or aligned in ways that make complex trade-offs unnecessary.

Key Results

Heuristics guided by cumulative dominance achieved a median accuracy of 96.1% (numeric) and 99.6% (binary) relative to the full linear model. Lexicographic rules (which stop at the first discriminating cue) were nearly as accurate as the base linear rule, sometimes even exceeding it in predictive accuracy due to lower variance and avoided overfitting.

Prevalence of dominance and cumulative dominance

In binary datasets, simple dominance was found in 51% of object pairs on average. Cumulative dominance, however, was found in a staggering 87% of pairs, explaining the high success rate of simplified decision rules.

Limitations and Future Work

  • The "Correct" Order: The study assumes the heuristic knows the correct cue directions and the correct order of importance (matching the linear model). If a heuristic uses a sub-optimal ordering, the environmental structures identified here may not be enough to save its performance.
  • Simplified Probabilistic Model: The probabilistic approximation of dominance assumes attributes are independent and uniformly distributed. Real-world attribute correlations might complicate this model.
  • Future Direction: The author suggests that "approximate dominance" could be turned into a new class of decision heuristics themselves, combining the simplicity of heuristics with the statistical rigor of probabilistic estimation.

Summary updated

Abstract

Many attempts to understand the success of simple decision heuristics have examined heuristics as an approximation to a linear decision rule. This research has identified three environmental structures that aid heuristics: dominance, cumulative dominance, and noncompensatoriness. Here, we further develop these ideas and examine their empirical relevance in 51 natural environments. We find that all three structures are prevalent, making it possible for some simple rules to reach the accuracy levels of the linear decision rule using less information.
BibTeX
@inproceedings{simsek2013linear,
  author = {Özgür Simsek},
  title = {{Linear decision rule as aspiration for simple decision heuristics}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2013},
  url = {https://openreview.net/forum?id=u-7iht8U4Yf},
}
Figure