New Asymptotic Limit Theory and Inference for Monotone Regression

Soham Mallick, Siddhaarth Sarkar, Arun Kumar KuchibhotlaarXiv 2023 · DOI · arXiv

Abstract: Nonparametric regression problems with qualitative constraints such as monotonicity or convexity are ubiquitous in applications. For example, in predicting the yield of a factory in terms of the number of labor hours, the monotonicity of the conditional mean function is a natural constraint. One can estimate a monotone conditional mean function using nonparametric least squares estimation, which involves no tuning parameters. Several interesting properties of the isotonic LSE are known including its rate of convergence, adaptivity properties, and pointwise asymptotic distribution. However, we believe that the full richness of the asymptotic limit theory has not been explored in the literature which we do in this paper. Moreover, the inference problem is not fully settled. In this paper, we present some new results for monotone regression including an extension of existing results to triangular arrays, and provide asymptotically valid confidence intervals that are uniformly valid over a large class of distributions.

Introduction

Monotone regression, or isotonic regression, is a powerful nonparametric tool used when the relationship between a covariate and a response is known to be non-decreasing. For instance, in manufacturing, factory yield is expected to increase with labor hours. Unlike standard parametric models, the Isotonic Least Squares Estimator (LSE) does not assume a specific functional form and requires no tuning parameters, such as bandwidth.

A monotone regression fit (orange line) to a scatter plot of data points.

Despite its popularity, statistical inference for the Isotonic LSE has remained a significant challenge. The estimator's convergence rate depends on the local "flatness" of the underlying function—a property typically unknown to the researcher. Standard inferential techniques like the bootstrap are known to fail in this context, and existing asymptotic theory was largely limited to fixed distributions. This paper addresses these gaps by developing a new asymptotic limit theory and a robust, tuning-free method for constructing confidence intervals.

Methods and Theoretical Framework

The authors extend the existing theory by adopting a triangular array setting. This framework allows the underlying distribution to change as the sample size $n$ increases, capturing a broader range of local behaviors than previous "fixed distribution" models.

New Limit Theory

The paper demonstrates that the pointwise asymptotic distribution of the Isotonic LSE is characterized as the slope of the Greatest Convex Minorant (GCM) of a "drifted" Brownian motion. The specific nature of this drift is determined by the local smoothness and curvature of the true regression function.

The full pipeline of monotone regression: (left) the fitted regression function, (middle) the Greatest Convex Minorant (GCM) process characterizing the limit, and (right) the resulting symmetric distribution of the estimator.

Symmetry and the HulC Method

A key theoretical contribution is the proof of median unbiasedness. The authors show that the limiting distribution of the Isotonic LSE is symmetric around zero under a wide variety of conditions (Theorem 3.1). This symmetry is the "secret sauce" for inference.

By leveraging this symmetry, the authors adapt the Hull-based Confidence intervals (HulC) method. HulC works by splitting the dataset into several independent batches, calculating the isotonic estimate for each batch, and forming a confidence interval using the minimum and maximum of these estimates. Because the method relies on symmetry rather than the exact convergence rate, it is adaptive—it automatically adjusts to unknown local smoothness without requiring user-specified tuning parameters.

Empirical Results

The proposed theory and the HulC inference method were validated through extensive simulations using synthetic datasets. The researchers tested several challenging scenarios, including:

  • Varying Flatness: Functions with polynomial behavior $|x|^\theta \text{sgn}(x)$ where $\theta$ ranged from 0.2 to 10.
  • Asymmetry and Heteroscedasticity: Testing robustness against non-normal error distributions (e.g., non-central $\chi^2$) and noise levels that vary with the input.

The results showed that the HulC method consistently maintained the target 95% coverage across these diverse settings. It outperformed standard subsampling techniques, which often failed when the regression function lacked sufficient smoothness. While "oracle" methods (which assume knowledge of the true flatness parameter) performed well, HulC achieved comparable results without such prior knowledge.

Conclusion

This work significantly advances the understanding of isotonic regression by providing a unified asymptotic theory that encompasses previous results as special cases. By proving the symmetry of the limiting distribution, the authors have unlocked a practical, tuning-free approach to inference via the HulC method. Future research will focus on extending these results to multivariate monotone regression and exploring the conjecture that the limiting distribution is always unimodal, which could lead to even more precise inferential tools.

Summary updated

Abstract

Nonparametric regression problems with qualitative constraints such as monotonicity or convexity are ubiquitous in applications. For example, in predicting the yield of a factory in terms of the number of labor hours, the monotonicity of the conditional mean function is a natural constraint. One can estimate a monotone conditional mean function using nonparametric least squares estimation, which involves no tuning parameters. Several interesting properties of the isotonic LSE are known including its rate of convergence, adaptivity properties, and pointwise asymptotic distribution. However, we believe that the full richness of the asymptotic limit theory has not been explored in the literature which we do in this paper. Moreover, the inference problem is not fully settled. In this paper, we present some new results for monotone regression including an extension of existing results to triangular arrays, and provide asymptotically valid confidence intervals that are uniformly valid over a large class of distributions.
BibTeX
@misc{mallick2023new,
  author = {Soham Mallick and Siddhaarth Sarkar and Arun Kumar Kuchibhotla},
  title = {{New Asymptotic Limit Theory and Inference for Monotone Regression}},
  year = {2023},
  eprint = {2310.20058},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2310.20058},
}

Loading related content...

Figure