Context Dependent Choice Models for Human Preferences

You are reading version 1 of this direction, frozen 2026-10-04. The latest version is version 2.

Version 1 of 2152 papers242 conceptsfrozen 2026-10-04

Most algorithms that predict what humans want—from recommendation engines to AI alignment via RLHF—rely on a mathematical convenience called the Independence of Irrelevant Alternatives (IIA). This principle assumes that if you prefer an apple over an orange, your preference shouldn't change if a banana is added to the bowl.

While IIA makes models like the Multinomial Logit (MNL) and Plackett-Luce computationally elegant, it fails to capture the messy reality of human psychology. Recent research is moving toward "context-dependent" models that recognize that the set of available options fundamentally reshapes how we value each individual choice.

The Failure of Independence

The classic intuition for why IIA fails is the Red Bus/Blue Bus paradox. Imagine you are choosing between a car and a red bus. If they are equally attractive, you have a 50/50 chance of picking either. If a blue bus is introduced—identical to the red one in every way except color—a standard IIA model would treat it as a brand-new alternative and split the probability three ways (33% each). In reality, you likely still have a 50% preference for the car and split the remaining 50% between the two buses.

As Panayotis Mertikopoulos and colleagues argue in Nested Bandits, treating similar items as independent leads to "perverse" exploration. Algorithms waste time testing redundant options, diluting the probability of selecting the truly optimal choice.

This isn't just a theoretical quirk; it has high-stakes consequences for AI. Wanqiao Xu and Benjamin Van Roy demonstrate in RLHF and IIA: Perverse Incentives that because RLHF typically uses pairwise comparisons (which inherently assume IIA), it can create incentives for LLMs to favor behaviors that humans don't actually prefer, simply because of how the queries are formatted.

Capturing Context Effects

If IIA is the problem, how do we fix it? Researchers are pursuing three main strategies to build context-aware models:

1. Hierarchical and Nested Structures

Instead of treating all options as a flat list, nested models group similar items together. In the bus example, a model would first choose between "Car" and "Bus," and then choose the color. Tien Anh Mai and Arunesh Sinha show in Choices Are Not Independent that these "Nested Quantal Response" models are more effective at predicting human adversaries in security games, where attackers perceive targets as related groups rather than isolated points.

2. Neural Context Architectures

Modern deep learning allows us to process the entire "menu" of options simultaneously. Shuhan Zhang and Rui Gao introduced DeepHalo, a neural choice model that explicitly learns "context effects" like the decoy effect (where an inferior third option makes a target look better). Similarly, Alejandro Mottini used Pointer Networks in Deep Choice Model to predict airline itinerary choices, outperforming industry standards by looking at the relative attributes of all flights at once.

3. Axiomatic Relaxation

In social choice and voting, researchers are weakening the IIA requirement to avoid the "impossibility" results of Arrow's Theorem. Wesley H. Holliday and Eric Pacuit propose "Coherent IIA" in Axioms for Defeat in Democratic Elections. This allows the relationship between two candidates to depend on the broader context—specifically whether a "cycle" of preferences exists—without allowing a spoiler candidate to flip the winner arbitrarily.

The Statistical Barrier

Moving beyond IIA comes with a steep cost: data complexity. Yeshwanth Cherapanamjeri and Constantinos Daskalakis prove in Learning Correlated Reward Models that standard pairwise data (A vs. B) is mathematically insufficient to learn how preferences are correlated. To unlock these more accurate models, we need richer data, such as "best-of-three" rankings, which reveal how the presence of a third item shifts the relative value of the first two.

Why It Works: Intuition vs. Absolute Utility

The core intuition behind this shift is that humans do not have "absolute utility" meters in their heads. Instead, we are comparative engines. As Haiyang Chen suggests in Implications of Human Irrationality for Reinforcement Learning, these "biases" might actually be computationally efficient heuristics for making decisions in noisy, uncertain environments. By modeling the context, we aren't just fixing "errors" in human logic—we are building AI that finally understands the relational nature of human value.

Papers in this version

Showing 60 of 152 papers.

History & citationVersion 2 · 2026-10-04

Current · Permanent link

  1. Version 2 (latest) · 2026-10-04 · 108 papers
  2. Version 1 · 2026-10-04 · 152 papers
Cite this direction

Latest version (a living page):

@misc{lacuna_independence_of_irrelevant_alternatives_adb840e3,
  title = {independence of irrelevant alternatives},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/independence-of-irrelevant-alternatives/rdr_adb840e31e43420f9789f2a8ab0289a1},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_context_dependent_choice_models_for_huma_adb840e3_v1,
  title = {Context Dependent Choice Models for Human Preferences},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/context-dependent-choice-models-for-human-preferences/rdr_adb840e31e43420f9789f2a8ab0289a1/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}