Nonseparable multinomial choice models in cross-section and panel data

Victor Chernozhukov, Iván Fernández‐Val, Whitney K. Newey

This paper, authored by Victor Chernozhukov, Iván Fernández-Val, and Whitney K. Newey, is a deep dive into the econometrics of how we model discrete choices—like choosing between brands of cereal or modes of transportation—when the underlying factors are complex and hidden.

TL;DR

The paper provides new theoretical tools to identify the impact of observed variables on consumer utility in "nonseparable" models, where hidden individual traits interact with observable features in complex ways. By analyzing the derivatives of choice probabilities, the authors show how to recover average utility effects in both snapshot (cross-section) data and over-time (panel) data.

What Problem Does This Paper Address?

In standard economic modeling, we often assume that unobserved factors (like a person's specific taste) simply add a bit of "noise" to a choice. This is called a "separable" model. However, the real world is rarely that simple. A person’s hidden risk aversion might change how they respond to a price increase; here, the unobserved trait interacts with the observed variable. This is a nonseparable model.

The challenge is "identification": can we actually figure out the true effect of a variable (like price or quality) on someone's utility if we can't see these complex hidden interactions? Previous research often relied on very restrictive assumptions. This paper aims to relax those restrictions, providing a mathematical framework to understand these effects using derivatives of observed choice probabilities.

Methods

The authors use advanced econometric theory to derive identification results. Their approach focuses on derivatives:

  • Cross-Sectional Analysis: They prove that the derivative of a choice probability (how much the likelihood of a choice changes when a variable changes) is actually a weighted average of the underlying utility derivatives.
  • Random Coefficients: They explore models where the "weights" people give to different attributes are themselves random variables, showing that at certain points (like zero), the probability derivative is directly proportional to the average of these random coefficients.
  • Control Functions: To handle "endogeneity" (when observed variables are correlated with unobserved ones, like high-end stores being placed in wealthy neighborhoods), they use control functions to "clean" the data and recover structural effects.
  • Panel Data & Time Stationarity: In data where we observe the same people over two periods, they assume that the distribution of hidden traits is stable over time. They use this "stationarity" to cancel out the bias caused by hidden factors.

Datasets

Not discussed in the paper. This is a purely theoretical and methodological paper. It focuses on proving mathematical theorems and providing formulas that other researchers can apply to their own empirical datasets. It does not use real-world data or introduce new benchmarks.

Key Insights and Ideas

  • The "Indifference" Filter: A key insight is that probability derivatives only capture the effects for "marginal" individuals—those who are currently indifferent between two choices. The math "filters" the population to look at these specific cases.
  • The Power of the "Diagonal": In panel data, the authors find that we can perfectly identify utility effects "on the diagonal"—that is, when a person's attributes are the same in both time periods.
  • The "Off-Diagonal" Limit: Perhaps the most surprising finding is a negative one: they prove that time stationarity cannot identify utility effects "off the diagonal" (when attributes change between periods). They show that two different, equally plausible models could produce the same data, making it impossible to distinguish between them without more data or stronger assumptions.
  • Simplifying Complexity: While the underlying models are nonseparable and multidimensional, the authors show that the resulting probability derivatives often have a surprisingly simple interpretation as weighted averages.

Key Results

  • Theorem 1 & 2: Establish that in binary choice models, the derivative of the choice probability is a scalar multiple of the average utility derivative at the "zero threshold" (the point of indifference).
  • Theorem 4: Extends these findings to multinomial choice (choosing between $J$ alternatives), showing the derivative is a sum of weighted utility effects across all options.
  • Theorem 9: Provides a specific identification result for multinomial panel data, showing that the vector of coefficients for continuous regressors is identified "up to scale" even if the data is bounded.
  • Non-Identification Proof: The paper formally demonstrates that for two-period panel data, the average utility derivative is fundamentally not identified when moving away from the "diagonal" where regressors are equal across periods.

Limitations and Future Work

  • Local Identification: Many of the results are "local," meaning they only tell us about people at the margin of making a different choice, rather than the entire population.
  • The Two-Period Constraint: The non-identification result "off the diagonal" suggests that two periods of panel data might not be enough for certain types of complex models.
  • Future Directions: The authors suggest that future work should look for the "minimal conditions" required to extend these average utility derivatives to larger, non-marginal populations. They also leave open the question of how to handle more than two periods of panel data to potentially overcome the "off-diagonal" identification gap.

Summary updated

Abstract

Multinomial choice models are fundamental for empirical modeling of economic choices among discrete alternatives. We analyze identification of binary and multinomial choice models when the choice utilities are nonseparable in observed attributes and multidimensional unobserved heterogeneity with cross-section and panel data. We show that derivatives of choice probabilities with respect to continuous attributes are weighted averages of utility derivatives in cross-section models with exogenous heterogeneity. In the special case of random coefficient models with an independent additive effect, we further characterize that the probability derivative at zero is proportional to the population mean of the coefficients. We extend the identification results to models with endogenous heterogeneity using either a control function or panel data. In time stationary panel models with two periods, we find that differences over time of derivatives of choice probabilities identify utility derivatives "on the diagonal," i.e. when the observed attributes take the same values in the two periods. We also show that time stationarity does not identify structural derivatives "off the diagonal" both in continuous and multinomial choice panel models.
BibTeX
@article{chernozhukov2018nonseparable,
  author = {Victor Chernozhukov and Iván Fernández‐Val and Whitney K. Newey},
  title = {{Nonseparable multinomial choice models in cross-section and panel data}},
  journal = {Journal of Econometrics},
  year = {2018},
  doi = {10.1016/j.jeconom.2018.12.008},
}

Loading related content...

Figure