Abstract: The most common approach to estimating conditional quantile curves is to fit a curve, typically linear, pointwise for each quantile. Linear functional forms, coupled with pointwise fitting, are used for a number of reasons including parsimony of the resulting approximations and good computational properties. The resulting fits, however, may not respect a logical monotonicity requirement that the quantile curve be increasing as a function of probability. This paper studies the natural monotonization of these empirical curves induced by sampling from the estimated non-monotone model, and then taking the resulting conditional quantile curves that by construction are monotone in the probability.
Introduction: The Quantile Crossing Problem
In econometrics and statistics, estimating conditional quantile functions is a standard task—for instance, determining the 90th percentile of income given a specific education level. By definition, these functions must be monotonic: a higher quantile must always be greater than or equal to a lower one. However, standard estimation techniques, such as linear quantile regression, often fit each quantile independently. Due to sampling noise or model misspecification, these estimated curves frequently cross each other, violating logical consistency. This "quantile crossing problem" complicates policy interpretation and decision-making.
The Rearrangement Operator
The core contribution of this work is the Rearrangement Operator, a method to restore monotonicity to non-monotone estimates. The intuition is straightforward: if an estimated curve wiggles up and down, it can be transformed into a monotone curve by "sorting" its values. Formally, the rearranged function is defined as the quantile function of the original estimated function when treated as a random variable.
As shown in the figure above, the dashed lines represent the initial non-monotone estimates, while the solid lines show the logically consistent, rearranged versions. This process ensures that the resulting curves respect the fundamental laws of probability.
Theoretical Foundations
The authors establish a rigorous framework for this operator using the Functional Delta Method. They prove that the rearrangement operation is Hadamard differentiable, which allows for a functional limit theory. This ensures that researchers can continue to use standard tools like confidence intervals and hypothesis testing on the "fixed" curves. Furthermore, the paper demonstrates that the bootstrap remains a valid method for uncertainty estimation after rearrangement.
Key Insights: Beyond a Cosmetic Fix
A critical finding is that rearrangement is not merely a cosmetic correction. The rearranged curve is mathematically guaranteed to be closer to the true monotone population curve (in $L^p$ norm) than the original non-monotone estimate.
Rearrangement vs. Isotonization
Traditionally, researchers used "isotonization" (like the pool-adjacent-violators algorithm) to enforce monotonicity. However, isotonization often creates artificial "flat spots" in the resulting curves. In contrast, rearrangement converts non-monotone segments into steep, strictly increasing segments. This behavior is often more appropriate for quantile functions, which are rarely flat in practice.
As illustrated by the derivative plots, the rearranged functions maintain positive slopes, avoiding the zero-derivative regions that characterize isotonized estimates.
Empirical Evidence and Performance
The method was evaluated using both empirical data and Monte Carlo simulations:
- Earnings Data: The authors applied the method to March Current Population Survey data (1979–1985) to study the impact of Vietnam veteran status on earnings, utilizing the U.S. draft lottery as an instrumental variable.
- Simulations: In synthetic "location model" experiments designed to mimic the veteran earnings data, rearranged estimators reduced estimation error by up to 14% compared to non-monotone estimates and outperformed isotonization by 3-4%.
Importantly, the authors show asymptotic equivalence: if the true population curve is already monotone, the rearranged estimate has the same first-order asymptotic distribution as the original in large samples. This implies there is "no cost" to applying the fix if the original model was already correct.
Conclusion
The rearrangement operator provides a simple, "plug-and-play" solution to the quantile crossing problem. By ensuring logical consistency while simultaneously improving estimation accuracy, it offers a robust tool for researchers working with conditional distributions and quantiles. Its ability to maintain valid inference through the functional delta method makes it a practical addition to the econometrician's toolkit.
Summary updated