Self Consistency: A General Recipe for Wavelet Estimation With Irregularly-spaced and/or Incomplete Data

Thomas C. M. Lee, Xiao‐Li MengarXiv 2007 · DOI · arXiv

Abstract

Inspired by the key principle behind the Expectation-Maximization (EM) algorithm, this paper proposes a general methodology for conducting wavelet estimation with irregularly spaced data by viewing the observations as the observed portion of an augmented, regularly spaced dataset. By invoking the self-consistency principle, we define wavelet estimators that function effectively in the presence of incomplete data. This approach offers several major advantages: it can be coupled with almost any wavelet shrinkage method, it accommodates non-Gaussian or correlated noise, and it automatically handles various types of missing observations. We develop a multiple-imputation algorithm (MISC) and fast EM-type algorithms (Sim and Ref) for computing these estimates. Numerical experiments demonstrate that these algorithms outperform several common methods in both 1D signal smoothing and 2D image denoising.

Introduction

Wavelet transform techniques have become a cornerstone of signal and image processing due to their ability to adapt to local features and provide sparse representations of complex data. However, standard wavelet methods typically require a rigid data structure: exactly $N = 2^J$ observations sampled at perfectly equal intervals. In practical applications—ranging from astronomy to medical imaging—sensors fail, pixels are corrupted, and sampling is frequently irregular.

Historically, researchers addressed these irregularities by pre-processing data via interpolation to force it onto a regular grid. However, these methods often introduce implicit smoothness assumptions that may not reflect the underlying signal or fail to account for the uncertainty introduced by the "filled-in" values. This paper introduces a mathematically principled framework to handle incomplete data without such rigid constraints.

The Self-Consistency Framework

The core innovation of this research is the treatment of an irregular design as a regular design with missing observations. By framing the problem this way, the authors apply the self-consistency principle, which is the foundational logic of the EM algorithm. An estimate is considered "self-consistent" if it equals the expected value of the "best" estimate one would obtain if the missing data were known.

To quantify the impact of missing data, the authors introduce $\eta_l$, a measure of "missingness" for each wavelet coefficient. This diagnostic tool allows the algorithm to identify which specific components of the signal—often the high-frequency details—are most compromised by the sampling pattern.

Algorithmic Implementations

The paper develops three primary algorithms to implement this self-consistency framework:

  1. MISC (Multiple Imputation Self-Consistent): A Monte Carlo approach that simulates multiple versions of the missing data, applies standard wavelet shrinkage to each, and averages the results. While statistically robust and flexible, it is computationally intensive.
  2. Sim (Simple Approximated): A high-speed variant that imputes missing values using the current signal estimate. It utilizes a variance inflation formula to adjust noise estimates, correcting for the fact that imputed values are inherently "too perfect" and lack the natural variance of observed data.
  3. Ref (Refined Fast): This algorithm represents the optimal balance between efficiency and accuracy. It uses an analytic closed-form approximation for the expectation step, providing the statistical efficiency of MISC with the computational speed of the Sim algorithm.

Experimental Results and Insights

Performance Benchmarks

The algorithms were evaluated using the standard "Big Four" 1D test functions (Bumps, Blocks, Doppler, and Heavisine). In comparisons with existing methods such as IRREGSURE and ROSE, the RefAI (Refined with Interpolation) algorithm consistently achieved lower Mean Squared Error (MSE). Notably, the framework remained robust even when 50% of the data was missing, significantly outperforming naive approaches.

Flexibility in Data Types

To demonstrate the method's versatility, the authors applied it to:

  • Astronomy: Smoothing Poisson photon counts from the star RXJ1856.5-3754, proving the method's efficacy with non-Gaussian noise.
  • Image Processing: Reconstructing the classic Lena and Airplane images. The 2D implementation successfully performed "inpainting," reconstructing images with clustered missing pixels and preventing damage from smearing across the reconstruction.

Key Theoretical Insights

  • Information Separation: The framework distinguishes between information derived from observed data and information added by external assumptions. This allows for "hybrid" models where interpolation is only used when the underlying signal is known to be smooth.
  • Threshold Transformation: A fascinating finding is that the self-consistency requirement naturally transforms "hard thresholding" into a form of soft thresholding when data is missing, providing a smoother transition in the estimation process.

Conclusion and Future Work

While this paper provides a robust "engineering" framework and strong empirical evidence, the authors note that rigorous theoretical proofs for the existence and uniqueness of the self-consistent solution in this context are still required. Future research directions include optimizing the computational cost of $\eta_l$ calculations for large-scale 2D images and integrating the self-consistency principle with Bayesian wavelet methods for improved parameter inference. Ultimately, this methodology provides a general, flexible recipe for wavelet estimation that bridges the gap between theoretical wavelet requirements and the messy reality of experimental data.

Summary updated

Abstract

Inspired by the key principle behind the EM algorithm, we propose a general methodology for conducting wavelet estimation with irregularly-spaced data by viewing the data as the observed portion of an augmented regularly-spaced data set. We then invoke the self-consistency principle to define our wavelet estimators in the presence of incomplete data. Major advantages of this approach include: (i) it can be coupled with almost any wavelet shrinkage methods, (ii) it can deal with non--Gaussian or correlated noise, and (iii) it can automatically handle other kinds of missing or incomplete observations. We also develop a multiple-imputation algorithm and fast EM-type algorithms for computing or approximating such estimates. Results from numerical experiments suggest that our algorithms produce favorite results when comparing to several common methods, and therefore we hope these empirical findings would motivate subsequent theoretical investigations. To illustrate the flexibility of our approach, examples with Poisson data smoothing and image denoising are also provided.
BibTeX
@misc{lee2007self,
  author = {Thomas C. M. Lee and Xiao‐Li Meng},
  title = {{Self Consistency: A General Recipe for Wavelet Estimation With Irregularly-spaced and/or Incomplete Data}},
  year = {2007},
  eprint = {math/0701196},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/math/0701196},
  howpublished = {arXiv},
}
Figure