Distribution Regression with Sample Selection, with an Application to Wage Decompositions in the UK

Victor Chernozhukov, Iván Fernández‐Val, Siyi LuoarXiv 2018 · DOI · arXiv

TL;DR

This paper introduces a semi-parametric generalization of the classic Heckman selection model using distribution regression. By allowing for non-Gaussian errors and heterogeneous effects across the entire wage distribution, the authors uncover that the UK gender wage gap (21–40%) is driven largely by differences in wage structures and complex "sorting" behaviors—specifically, positive selection for single men and negative selection for married women.

What Problem Does This Paper Address?

In labor economics, we rarely observe the "offered wage" for everyone—only for those who actually choose to work. This creates sample selection bias. If the people who choose to work are systematically different (e.g., more motivated or highly skilled) than those who don't, simply comparing average wages between groups leads to "garbage-in, garbage-out" results.

The gold standard for fixing this has long been the Heckman selection model. However, the classic Heckman model relies on two very stiff assumptions:

  1. Gaussianity: It assumes the unobserved factors (shocks) follow a normal distribution.
  2. Homogeneity: It assumes that covariates (like education) shift the mean of the wage but don't change the shape of the wage distribution.

In reality, the "glass ceiling" or "sticky floors" suggest that gender discrimination and selection work differently at the 10th percentile than at the 90th. This paper provides a flexible, semi-parametric framework to model these "distributional" effects without being handcuffed by the assumption of normality.

Methods

The authors develop a Distribution Regression (DR) model with endogenous sample selection. The technical "secret sauce" is the Local Gaussian Representation (LGR).

Instead of assuming a global bivariate normal distribution, they represent the joint distribution of the outcome (wage) and selection (employment) at every point $(y, d)$ using a bivariate normal CDF, but with a local correlation parameter $\rho(y, d)$. This allows the "strength of selection" to vary across the distribution.

The estimation follows a tractable two-step approach:

  1. Selection Step: A simple probit model to estimate the probability of being employed.
  2. Outcome Step: Multiple bivariate probit regressions with sample selection corrections to estimate the entire conditional distribution of wages.

To handle the heavy lifting of statistical significance, they use a multiplier bootstrap to generate uniform confidence bands, allowing researchers to test hypotheses about the entire wage function at once.

Datasets

The authors apply their model to a massive dataset of UK wages spanning 36 years (1978–2013). They harmonize data from three major surveys:

  • Family Expenditure Survey (FES)
  • Expenditure and Food Survey (EFS)
  • Living Costs and Food Survey (LCFS)

The final sample includes approximately 258,900 observations (139,504 women and 119,396 men), focusing on individuals aged 23 to 59.

Key Insights and Ideas

  • The "Sorting" Signature: The paper distinguishes between "selection structure" (how observable traits affect employment) and "selection sorting" (how unobservable traits correlate between employment and wages).
  • Positive vs. Negative Sorting: They find positive sorting for single men (those with high unobserved wage potential are more likely to work) but negative sorting for married women.
  • The Marriage Market & Glass Ceilings: The negative sorting for married women is a major "aha" moment. It suggests that women with very high potential wages might opt out of the labor force. This is consistent with assortative matching (high-earning women married to high-earning men who can afford to stay home) or the glass ceiling (high-potential women opting out because top-tier jobs are inaccessible).
  • Location Shifting is Not Enough: The data firmly reject the "location shift" restriction of the classical Heckman model. Covariates like marital status impact the shape and spread of the wage distribution, not just the average.

Key Results

  • The Latent Gap: After controlling for selection, the "offered" gender wage gap remains substantial, ranging from 21% to 40% across the distribution.
  • Wage Structure is King: Most of the gap is explained by the "wage structure" (the returns to characteristics), which is often interpreted as a proxy for labor market discrimination.
  • Sorting Explains the Top: At the top of the wage distribution, the difference in selection sorting between men and women explains a significant portion of the observed gap. Specifically, negative sorting for women accounts for 30–40% of the gap at the upper tail.
  • Education: The positive effect of education on wages is not uniform; it increases as you move up the wage distribution.

Limitations and Future Work

  • Work Experience: A notable limitation is the lack of a direct measure of work experience in the UK datasets used. While the authors use age as a proxy and perform robustness checks, experience is a critical driver of wages.
  • Instrument Validity: The model relies on "out-of-work benefit income" as an instrumental variable (exclusion restriction). While standard in the literature, the validity of this instrument is often debated among economists.
  • Dynamic Effects: The current model is a cross-sectional snapshot. Future work could extend this to panel data to see how distributional selection evolves over an individual's lifecycle.

Reference: Chernozhukov, V., Fernández-Val, I., & Luo, S. (2023). Distribution Regression with Sample Selection, with an Application to Wage Decompositions in the UK. arXiv:1811.11603v6 [econ.EM].

Summary updated

Abstract

We develop a distribution regression model under endogenous sample selection. This model is a semi-parametric generalization of the Heckman selection model. It accommodates much richer effects of the covariates on outcome distribution and patterns of heterogeneity in the selection process, and allows for drastic departures from the Gaussian error structure, while maintaining the same level tractability as the classical model. The model applies to continuous, discrete and mixed outcomes. We provide identification, estimation, and inference methods, and apply them to obtain wage decomposition for the UK. Here we decompose the difference between the male and female wage distributions into composition, wage structure, selection structure, and selection sorting effects. After controlling for endogenous employment selection, we still find substantial gender wage gap -- ranging from 21% to 40% throughout the (latent) offered wage distribution that is not explained by composition. We also uncover positive sorting for single men and negative sorting for married women that accounts for a substantive fraction of the gender wage gap at the top of the distribution.
BibTeX
@misc{chernozhukov2018distribution,
  author = {Victor Chernozhukov and Iván Fernández‐Val and Siyi Luo},
  title = {{Distribution Regression with Sample Selection, with an Application to Wage Decompositions in the UK}},
  year = {2018},
  eprint = {1811.11603},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1811.11603},
}

Loading related content...

Figure