Asymptotics for change-point models under varying degrees of mis-specification

Rui Song, Moulinath Banerjee, Michael R. Kosorok

TL;DR

This paper provides a theoretical "bridge" between two extremes in change-point analysis: when a jump model perfectly fits the data versus when it is fitted to a completely smooth curve. The authors uncover a spectrum of asymptotic behaviors—including a new "intermediate" regime—and provide an adaptive subsampling method to perform reliable statistical inference even when the true underlying model is unknown.

What Problem Does This Paper Address?

In statistics, a change-point model is used to describe data that undergoes an abrupt shift (like a sudden jump in a stock price or a phase transition in physics). Traditionally, theorists have studied two scenarios:

  1. Correct Specification: The data actually has a jump. Here, estimators are very efficient, converging at a fast rate ($n$).
  2. Complete Mis-specification: You fit a jump model to data that is actually a fixed, smooth curve. In this case, the estimator is much "lazier," converging at a much slower rate ($n^{1/3}$).

The problem is that real-world data often falls somewhere in between. A signal might look like a sharp jump in a small sample but reveal itself as a very steep, smooth curve as more data arrives. Before this paper, there was no unified theory to explain how the transition from "smooth" to "jump" happens and how it affects our ability to estimate where that change occurs.

Methods

The authors study a sequence of models where a smooth function $f_n$ approaches a "stump" (a single-jump function) as the sample size $n$ increases. The "sharpness" of this transition is controlled by a parameter $\alpha_n$.

They employ Least Squares Estimation (LSE) to find the best-fitting jump model. To analyze the behavior, they break the problem down into three distinct asymptotic regimes based on how fast the smooth curve "sharpens" relative to the sample size:

  • The Slow Regime ($\alpha_n = o(n)$): The curve sharpens slowly.
  • The Intermediate Regime ($\alpha_n = n$): The sharpening matches the growth of data.
  • The Fast Regime ($n = o(\alpha_n)$): The curve sharpens so quickly it effectively looks like a jump.

Crucially, they introduce an adaptive inference procedure. Since a practitioner doesn't know which regime their data belongs to, the authors use a subsampling technique (a variation of the bootstrap) to estimate the convergence rate directly from the data, allowing for the construction of valid confidence intervals.

Datasets

As a theoretical statistics paper, the work relies on carefully designed simulations rather than standard machine learning benchmarks.

  • Synthetic Models: They generated data using sigmoidal functions (logistic-type curves) where the steepness ($M$ or $\alpha_n$) was varied.
  • Noise and Covariates: They used Gaussian noise and uniform covariates to test the robustness of their estimators across different sample sizes (ranging from $n=50$ to $n=4000$).
  • Comparison: They compared their empirical results against the theoretical "Chernoff limit" (for smooth models) and "Compound Poisson limits" (for jump models).

Key Insights and Ideas

  • The Convergence Spectrum: The most important "aha moment" is that the rate of convergence isn't just $n$ or $n^{1/3}$; it is a continuous spectrum. The rate depends entirely on the "local" smoothness of the curve near the suspected change-point.
  • The Intermediate Limit: The authors discovered a new family of "boundary distributions" for the $\alpha_n = n$ case. This distribution is a hybrid that can qualitatively transition into either the Gaussian-based limits of smooth models or the jump-process limits of discrete models.
  • Resolution vs. Reality: At small sample sizes, a very steep smooth curve is indistinguishable from a jump (the "Fast Regime" is the best approximation). As $n$ grows, the estimator eventually "sees" the smoothness, and the "Slow Regime" (Chernoff limit) becomes the more accurate description.

Key Results

  • Regime Identification:
    • In the Fast Regime, the estimator converges at rate $n$, and the limit is the minimizer of a compound Poisson process.
  • In the Slow Regime, the rate is $n^{1/3}$, and the limit is a rescaled Chernoff distribution.
  • Adaptive Inference: Their 5-step adaptive subsampling procedure successfully produced confidence intervals with near-target coverage (e.g., ~95%) across all three regimes, proving that you don't need to know the "true" degree of mis-specification to do good science.
  • Correlation Shifts: They showed that in the smooth (slow) regime, estimates of the jump levels and the jump location are asymptotically correlated, whereas in the jump (fast) regime, they become independent.

Limitations and Future Work

  • Multiple Change-Points: This paper focuses on a single "stump" (one jump). Extending this to multiple change-points would require handling complex interactions between different local sharpening rates.
  • Tuning Parameters: The adaptive subsampling method requires choosing subsample sizes ($n_1, n_2$). While the paper provides heuristics, the authors note that the reliability of results can be sensitive to these choices.
  • Smoothed LSE: The authors suggest that using a "smoothed" version of the least squares criterion (using kernels) might lead to more standard (Normal) distributions, though at the cost of even slower convergence rates. This remains a promising area for future comparison.

Summary updated

Abstract

Change-point models are widely used by statisticians to model drastic changes in the pattern of observed data. Least squares/maximum likelihood based estimation of change-points leads to curious asymptotic phenomena. When the change-point model is correctly specified, such estimates generally converge at a fast rate ($n$) and are asymptotically described by minimizers of jump process. Under complete mis-specification by a smooth curve, i.e. when a change-point model is fitted to data described by a smooth curve, the rate of convergence slows down to $n^{1/3}$ and the limit distribution changes to that of the minimizer of a continuous Gaussian process. In this paper we provide a bridge between these two extreme scenarios by studying the limit behavior of change-point estimates under varying degrees of model mis-specification by smooth curves, which can be viewed as local alternatives. We find that the limiting regime depends on how quickly the alternatives approach a change-point model. We unravel a family of `intermediate' limits that can transition, at least qualitatively, to the limits in the two extreme scenarios.
BibTeX
@article{song2015asymptotics,
  author = {Rui Song and Moulinath Banerjee and Michael R. Kosorok},
  title = {{Asymptotics for change-point models under varying degrees of mis-specification}},
  journal = {The Annals of Statistics},
  year = {2015},
  doi = {10.1214/15-aos1362},
}

Loading related content...

Figure