Quantile and probability curves without crossing

Iván Fernández‐Val, Victor Chernozhukov, Alfred Galichon, Esther DufloWorking 2017 · DOI · arXiv · OpenAlex

Abstract: The most common approach to estimating conditional quantile curves is to fit a curve, typically linear, pointwise for each quantile. Linear functional forms, coupled with pointwise fitting, are used for a number of reasons including parsimony of the resulting approximations and good computational properties. The resulting fits, however, may not respect a logical monotonicity requirement that the quantile curve be increasing as a function of probability. This paper studies the natural monotonization of these empirical curves induced by sampling from the estimated non-monotone model, and then taking the resulting conditional quantile curves that by construction are monotone in the probability.

Introduction: The Quantile Crossing Problem

In econometrics and statistics, estimating conditional quantile functions is a standard task—for instance, determining the 90th percentile of income given a specific education level. By definition, these functions must be monotonic: a higher quantile must always be greater than or equal to a lower one. However, standard estimation techniques, such as linear quantile regression, often fit each quantile independently. Due to sampling noise or model misspecification, these estimated curves frequently cross each other, violating logical consistency. This "quantile crossing problem" complicates policy interpretation and decision-making.

The Rearrangement Operator

The core contribution of this work is the Rearrangement Operator, a method to restore monotonicity to non-monotone estimates. The intuition is straightforward: if an estimated curve wiggles up and down, it can be transformed into a monotone curve by "sorting" its values. Formally, the rearranged function is defined as the quantile function of the original estimated function when treated as a random variable.

Rearrangement of non-monotone quantile ($Q$) and distribution ($Q^{-1}$) functions into monotone counterparts ($Q^*$, $F$).

As shown in the figure above, the dashed lines represent the initial non-monotone estimates, while the solid lines show the logically consistent, rearranged versions. This process ensures that the resulting curves respect the fundamental laws of probability.

Theoretical Foundations

The authors establish a rigorous framework for this operator using the Functional Delta Method. They prove that the rearrangement operation is Hadamard differentiable, which allows for a functional limit theory. This ensures that researchers can continue to use standard tools like confidence intervals and hypothesis testing on the "fixed" curves. Furthermore, the paper demonstrates that the bootstrap remains a valid method for uncertainty estimation after rearrangement.

Key Insights: Beyond a Cosmetic Fix

A critical finding is that rearrangement is not merely a cosmetic correction. The rearranged curve is mathematically guaranteed to be closer to the true monotone population curve (in $L^p$ norm) than the original non-monotone estimate.

Rearrangement vs. Isotonization

Traditionally, researchers used "isotonization" (like the pool-adjacent-violators algorithm) to enforce monotonicity. However, isotonization often creates artificial "flat spots" in the resulting curves. In contrast, rearrangement converts non-monotone segments into steep, strictly increasing segments. This behavior is often more appropriate for quantile functions, which are rarely flat in practice.

Derivatives of the rearranged quantile ($\partial_u Q^*$) and distribution ($\partial_y F$) functions.

As illustrated by the derivative plots, the rearranged functions maintain positive slopes, avoiding the zero-derivative regions that characterize isotonized estimates.

Empirical Evidence and Performance

The method was evaluated using both empirical data and Monte Carlo simulations:

  • Earnings Data: The authors applied the method to March Current Population Survey data (1979–1985) to study the impact of Vietnam veteran status on earnings, utilizing the U.S. draft lottery as an instrumental variable.
  • Simulations: In synthetic "location model" experiments designed to mimic the veteran earnings data, rearranged estimators reduced estimation error by up to 14% compared to non-monotone estimates and outperformed isotonization by 3-4%.

Importantly, the authors show asymptotic equivalence: if the true population curve is already monotone, the rearranged estimate has the same first-order asymptotic distribution as the original in large samples. This implies there is "no cost" to applying the fix if the original model was already correct.

Conclusion

The rearrangement operator provides a simple, "plug-and-play" solution to the quantile crossing problem. By ensuring logical consistency while simultaneously improving estimation accuracy, it offers a robust tool for researchers working with conditional distributions and quantiles. Its ability to maintain valid inference through the functional delta method makes it a practical addition to the econometrician's toolkit.

Summary updated

Abstract

The most common approach to estimating conditional quantile curves is to fit a curve, typically linear, pointwise for each quantile. Linear functional forms, coupled with pointwise fitting, are used for a number of reasons including parsimony of the resulting approximations and good computational properties. The resulting fits, however, may not respect a logical monotonicity requirement that the quantile curve be increasing as a function of probability. This paper studies the natural monotonization of these empirical curves induced by sampling from the estimated non-monotone model, and then taking the resulting conditional quantile curves that by construction are monotone in the probability.
BibTeX
@inproceedings{fernandezval2017quantile,
  author = {Iván Fernández‐Val and Victor Chernozhukov and Alfred Galichon and Esther Duflo},
  title = {{Quantile and probability curves without crossing}},
  booktitle = {Working},
  year = {2017},
  doi = {10.1920/wp.cem.2007.1007},
}

Loading related content...

Figure