Predicting Choice with Set-Dependent Aggregation

Nir Rosenfeld, Kojin Oshiba, Yaron Singer

Abstract

Providing users with alternatives to choose from is an essential component in many online platforms, making the accurate prediction of choice vital to their success. A renewed interest in learning choice models has led to significant progress in modeling power, but most current methods are either limited in the types of choice behavior they capture, cannot be applied to large-scale data, or both. Here we propose a learning framework for predicting choice that is accurate, versatile, theoretically grounded, and scales well. Our key modeling point is that to account for how humans choose, predictive models must capture certain set-related invariances. Building on recent results in economics, we derive a class of models that can express any behavioral choice pattern, enjoy favorable sample complexity guarantees, and can be efficiently trained end-to-end. Experiments on three large choice datasets demonstrate the utility of our approach.

Introduction

Predicting user choices from a menu of options—such as flights, hotels, or news articles—is a critical task for modern online platforms. Traditional modeling approaches typically assign a score to each item independently and predict that the user will select the item with the highest score. However, this approach is fundamentally limited by the Independence of Irrelevant Alternatives (IIA) axiom. IIA assumes that the relative preference between two items remains unchanged regardless of other available options. In practice, human behavior frequently violates this principle; for instance, the introduction of a high-priced "decoy" can significantly increase the attractiveness of a middle-tier option.

Existing models that attempt to move beyond IIA often suffer from rigidity, poor scalability, or a focus on theoretical properties over predictive performance. This paper introduces Set-Dependent Aggregation (SDA), a framework designed to learn real-world IIA violations directly from data while maintaining the efficiency required for large-scale applications.

Methodology: Set-Dependent Aggregation

The core innovation of SDA is that it scores items based on the context of the entire set of available options rather than in isolation. This is achieved through two primary mechanisms:

  1. Set-Dependent Weights (SDW): The importance of an item's features is dynamically adjusted based on the composition of the set.
  2. Set-Dependent Embeddings (SDE): The latent representation of an item shifts according to its surrounding context.

Behavioral Inductive Bias

To prevent overfitting in unconstrained neural networks, the authors incorporate inductive biases from behavioral decision theory into the SDA architecture:

  • Relativity: Items are evaluated relative to a set-specific "reference point" generated by a permutation-invariant set neural network.
  • Asymmetry: Inspired by Prospect Theory, the model uses a custom "kinked tanh" activation function to reflect how humans perceive gains and losses differently.
  • Integrability: Multiple valuations are aggregated into a single choice using context-dependent weights.
The SDA architecture incorporates behavioral principles like reference points and asymmetric perception (kinked tanh) into a scalable neural network.

The authors demonstrate that SDA can theoretically approximate any form of IIA violation and provide Rademacher-based generalization bounds to ensure its learnability.

Experimental Evaluation

The model was evaluated on three large-scale, real-world click prediction datasets: Amadeus (flight itineraries), Expedia (hotel reservations), and Outbrain (news articles).

Key Results

SDA consistently outperformed traditional IIA-compliant baselines (e.g., Multinomial Logit, SVMRank) and existing non-IIA models (e.g., Mixed MNL, DeepSets) across all datasets. For example, on the Amadeus dataset, SDA achieved a Top-1 accuracy of 45.42%, a significant improvement over the standard Multinomial Logit (38.42%) and the more flexible but less constrained DeepSets (40.36%).

Performance comparison across three large-scale datasets shows SDA consistently outperforming both traditional and modern baselines.

Ablation studies confirmed that removing specific behavioral components—such as the kinked tanh activation or the set-dependent reference point—led to measurable performance declines, validating the importance of these psychology-inspired architectural choices.

Discussion and Insights

A key finding of this work is the "Goldilocks" zone of flexibility. While highly flexible models like DeepSets can theoretically capture any set function, they often overfit by finding false patterns in IIA violations. By contrast, SDA uses its "violation budget" more effectively by constraining the model with economic principles. The theoretical analysis suggests that the model effectively isolates specific regions of choice sets (e.g., those containing decoys) to apply its expressive power where it is most needed.

Conclusion and Future Work

SDA provides a scalable and theoretically grounded framework for choice prediction that bridges the gap between behavioral economics and deep learning. While the current work focuses on discrete choice tasks (selecting one item from a set), future research will explore extending SDA to sequential selection and subset selection tasks. Additionally, the authors propose investigating "violation complexity" as a metric to further understand the limits of learning complex choice behaviors.

Summary updated

Abstract

Providing users with alternatives to choose from is an essential component in many online platforms, making the accurate prediction of choice vital to their success. A renewed interest in learning choice models has led to significant progress in modeling power, but most current methods are either limited in the types of choice behavior they capture, cannot be applied to large-scale data, or both. Here we propose a learning framework for predicting choice that is accurate, versatile, theoretically grounded, and scales well. Our key modeling point is that to account for how humans choose, predictive models must capture certain set-related invariances. Building on recent results in economics, we derive a class of models that can express any behavioral choice pattern, enjoy favorable sample complexity guarantees, and can be efficiently trained end-to-end. Experiments on three large choice datasets demonstrate the utility of our approach.
BibTeX
@inproceedings{rosenfeld2020predicting,
  author = {Nir Rosenfeld and Kojin Oshiba and Yaron Singer},
  title = {{Predicting Choice with Set-Dependent Aggregation}},
  booktitle = {International Conference on Machine Learning},
  year = {2020},
  url = {https://openreview.net/forum?id=Zf4j3dcTB21},
}
Figure