Adaptive nonparametric estimation of a component density in a two-class mixture model

Gaëlle Chagny, Antoine Channarond, Van Hà Hoang, Angelina Roche

Introduction

In many scientific domains, such as genomics, signal processing, and astrophysics, researchers are often faced with the challenge of extracting a specific signal from a noisy background. This problem is frequently modeled using a "two-class mixture model," where observations are assumed to come from a combination of two sources: a known "null" distribution (representing noise or inactive states) and an unknown "alternative" distribution (representing the signal of interest).

This paper presents a robust statistical framework for the nonparametric adaptive estimation of the unknown component density. The authors introduce a "randomly weighted kernel estimator" (AWKE) that effectively disentangles the signal from the mixture without requiring prior knowledge of the signal's smoothness or the exact mixing proportions.

The Two-Class Mixture Model

The fundamental challenge addressed in this research is the estimation of a density $f$ from observations $X_1, ..., X_n$ that follow a mixture density $g$ defined as:

$$g(x) = \theta \phi(x) + (1 - \theta)f(x)$$

In this equation, $\phi(x)$ is a known probability density (often a uniform distribution in p-value analysis), $\theta \in [0, 1]$ is the unknown mixing proportion, and $f(x)$ is the unknown density of interest. The difficulty is twofold: first, the signal $f$ is observed only indirectly through the mixture $g$; second, the smoothness of $f$ is unknown, making the selection of an appropriate smoothing parameter (bandwidth) for kernel estimation particularly difficult.

Methodology: The Randomly Weighted Kernel Estimator

The Weighting Mechanism

The core innovation of the paper is the development of the Randomly Weighted Kernel Estimator (AWKE). Traditional kernel density estimators (KDE) are designed for direct observations. To adapt this to a mixture model, the authors derive a weight function based on a "change of density" principle. By weighting each observation $X_i$ by a factor that accounts for the local density of the mixture versus the known null, they create an estimator whose expected value aligns with the target density $f$.

Because the mixing proportion $\theta$ and the total mixture density $g$ are unknown in practice, the authors employ a "plug-in" approach. They use preliminary estimates—$\tilde{\theta}_n$ and $\hat{g}$—to calculate these weights dynamically from the data.

Adaptive Bandwidth Selection

A critical component of any kernel method is the bandwidth $h$, which controls the trade-off between bias (oversmoothing) and variance (undersmoothing). Usually, the optimal $h$ depends on the unknown smoothness of $f$. To overcome this, the authors implement the Goldenshluger-Lepski (GL) method. This fully data-driven procedure compares estimators across a grid of potential bandwidths and selects the one that minimizes a specific risk criterion, effectively "adapting" to the data's inherent structure without manual tuning.

Identifiability and Boundary Correction

To ensure the model is identifiable (i.e., that $f$ and $\theta$ can be uniquely determined), the authors assume that the signal $f$ vanishes in a certain region of the domain, typically near the boundary $x=1$. This allows for the estimation of $\theta$ by observing the behavior of the mixture where only the null component is present. Furthermore, to mitigate "boundary effects" common in kernel estimation, they utilize a reflection method to symmetrize the data near the edges of the support.

Theoretical Guarantees

The paper provides rigorous mathematical support for the AWKE method:

  • Oracle Inequality: The authors prove that their data-driven bandwidth selection performs nearly as well as an "oracle" that knows the perfect bandwidth in advance.
  • Convergence Rates: The estimator is shown to achieve the optimal minimax convergence rate of $(\frac{\log n}{n})^{2\beta/(2\beta+1)}$ for a function with $\beta$ levels of smoothness.
  • Robustness: A significant theoretical contribution is the proof that the errors introduced by using estimated weights (rather than true weights) do not degrade the final convergence rate of the estimator.

Numerical Results

The methodology was validated using simulated datasets featuring three distinct shapes for the unknown density $f$: polynomial decay, a truncated power distribution, and a truncated exponential distribution. The simulations varied the mixing proportion $\theta$ (0.35 to 0.65) and sample sizes $n$ (500 to 2000).

Key findings from the numerical study include:

  1. Superior Accuracy: The AWKE method consistently outperformed existing state-of-the-art estimators (such as the Nguyen and Matias estimator) in terms of Mean Squared Error (MSE).
  2. Stability: The method showed significantly better stability at the boundaries of the domain ($x=0.1$ and $x=0.9$) compared to previous weighted kernel approaches.
  3. Flexibility: The estimator successfully adapted to different functional forms, maintaining sharpness for exponential signals and smoothness for polynomial ones without manual intervention.

Conclusion and Future Work

This research provides a powerful tool for nonparametric density estimation in contaminated data environments. By combining randomly weighted kernels with the Goldenshluger-Lepski adaptation method, the authors have created an estimator that is both theoretically optimal and practically robust.

While the current work focuses on a uniform null distribution, the framework is designed to be extensible. Future research may explore the application of this method to more complex, non-uniform known components and investigate ways to reduce the computational overhead associated with the pairwise comparisons required by the GL bandwidth selection process.

Summary updated

Abstract

A two-class mixture model, where the density of one of the components is known, is considered. We address the issue of the nonparametric adaptive estimation of the unknown probability density of the second component. We propose a randomly weighted kernel estimator with a fully data-driven bandwidth selection method, in the spirit of the Goldenshluger and Lepski method. An oracle-type inequality for the pointwise quadratic risk is derived as well as convergence rates over Holder smoothness classes. The theoretical results are illustrated by numerical simulations.
BibTeX
@article{chagny2021adaptive,
  author = {Gaëlle Chagny and Antoine Channarond and Van Hà Hoang and Angelina Roche},
  title = {{Adaptive nonparametric estimation of a component density in a two-class mixture model}},
  journal = {Journal of Statistical Planning and Inference},
  year = {2021},
  doi = {10.1016/j.jspi.2021.05.004},
}
Figure