Bayesian wavelet-based signal estimation using non-informative priors

Mário A. T. Figueiredo, Robert D. Nowak2002 · DOI

TL;DR

This paper introduces a novel, computationally efficient, and translation-invariant (TI) wavelet-based signal denoising method. By leveraging non-informative Bayesian priors, it achieves state-of-the-art performance without the need for arbitrary parameter tuning, overcoming limitations of existing methods.

What Problem Does This Paper Address?

Signal denoising is a crucial task in many fields, and the Discrete Wavelet Transform (DWT) has proven highly effective due to its ability to represent signals sparsely and decorrelate them. However, existing DWT-based denoising methods often suffer from:

  • Arbitrary Thresholding: Many methods rely on ad hoc threshold levels or non-linear shrinkage functions that require manual tuning for optimal results.
  • Computational Expense: Signal-adaptive schemes, while potentially more accurate, can be computationally demanding.
  • Translation Variance: The DWT is not translation-invariant, meaning that shifting the input signal can lead to significantly different denoising results, often introducing undesirable artifacts like "pseudo-Gibbs" or "blocking."

While translation-invariant (TI) denoising approaches exist (typically by averaging results from multiple shifts), they often lack a strong theoretical foundation. This paper aims to address these issues by developing a principled, Bayesian approach to TI denoising that is both theoretically sound and practically efficient.

Methods

The authors propose a new wavelet-based signal denoising technique grounded in Bayesian inference and the theory of non-informative priors. Key aspects of their approach include:

  • Bayesian Formulation: The denoising problem is framed within a Bayesian context, where the unknown signal is modeled with a prior distribution, and observations are modeled with a likelihood function. Bayes' theorem is used to derive the posterior distribution, from which an optimal estimate can be obtained based on a chosen loss function.
  • Non-Informative Priors: To mitigate the arbitrariness of existing methods, the authors introduce a novel non-informative prior for the wavelet coefficients. This prior is designed to be scale-invariant and avoids introducing free parameters that require tuning. Specifically, they use a hierarchical Bayesian model with a Jeffreys prior on the variance of the wavelet coefficients.
  • Translation-Invariant (TI) Denoising: The paper derives a formal Bayesian criterion for TI denoising. This involves treating the signal shift as an unknown and using a uniform (non-informative) prior over all possible shifts. The resulting TI estimator is shown to be a weighted average of shift-dependent estimators, where the weights are determined by the evidence supporting each shift.
  • Empirical Bayes Approach: To handle the improper posterior distribution arising from the non-informative prior, the authors adopt an empirical Bayes approach. This involves estimating certain parameters (like the variance of the wavelet coefficients) from the data, effectively breaking the fully Bayesian chain.
  • Novel Shrinkage/Thresholding Rule: The proposed Bayesian framework leads to a simple, fixed non-linear shrinkage/thresholding rule for individual wavelet coefficients. This rule is shown to behave like soft thresholding for small coefficients and hard thresholding for large ones, offering a balance between the two classical approaches.

Datasets

The paper evaluates its proposed method using standard test signals commonly employed in signal processing research for denoising benchmarks. These include:

  • Blocks
  • Doppler
  • HeaviSine
  • Bumps

These signals are used to compare the performance of the new algorithm against existing methods across various Signal-to-Noise Ratios (SNRs). No new datasets or benchmarks are introduced.

Key Insights and Ideas

  • Principled TI Denoising: The paper provides the first formal estimation-theoretic derivation of translation-invariant (TI) denoising within a Bayesian framework, moving beyond ad hoc averaging methods.
  • Non-Informative Priors for Simplicity and Performance: By using non-informative priors, the authors achieve a denoising rule with no free parameters to tune, making it easy to apply and computationally efficient, while still outperforming more complex adaptive methods.
  • A Balanced Shrinkage Rule: The derived shrinkage/thresholding rule offers a compelling compromise between the aggressive shrinkage of soft thresholding and the more abrupt cutoff of hard thresholding, leading to improved denoising performance.
  • Scale Invariance is Key: The emphasis on scale-invariant priors is highlighted as a desirable property that ensures the denoising results are robust to changes in measurement units or signal scaling.

Key Results

  • Superior Performance: The proposed Bayesian wavelet-based denoising method consistently outperforms VisuShrink and SureShrink across various test signals and SNRs.
  • Competitive with State-of-the-Art: The new method achieves performance comparable to, and in some cases better than, computationally more demanding empirical Bayesian methods (e.g., those using Gaussian mixtures).
  • Computational Efficiency: Despite its advanced theoretical underpinnings, the proposed method is as computationally inexpensive as simpler schemes like VisuShrink.
  • Reduced Artifacts: The TI nature of the proposed estimator, as demonstrated in Figure 4, leads to a visually cleaner denoised signal with fewer artifacts compared to non-TI methods.

Limitations and Future Work

  • Empirical Bayes vs. Full Bayes: The authors acknowledge that their empirical Bayes approach breaks the fully Bayesian inference chain. While this simplifies computation, it means the derived TI estimator is not strictly optimal in a full Bayesian sense.
  • Estimating Shift Probabilities: For the theoretically optimal TI estimator (Equation 14), calculating the evidence $p(s|y)$ (the weights for averaging shift-dependent estimates) is challenging, especially within an empirical Bayes framework. The authors state they are currently investigating methods for estimating these probabilities.
  • Known Noise Variance: The formulation assumes the noise variance $\sigma^2$ is known. While they mention that good estimates can be obtained (e.g., from the MAD scheme), this is a common assumption in many denoising methods.

Future work will likely focus on developing robust methods for estimating the shift probabilities $p(s|y)$ to achieve the fully Bayesian optimal TI estimator and exploring the application of these techniques to other signal processing tasks.

Summary updated

Abstract

The sparseness and decorrelation properties of the discrete wavelet transform have been exploited to develop powerful signal denoising methods. Most existing schemes involve arbitrary thresholding nonlinearities and ad hoc threshold levels, or computationally expensive signal-adaptive procedures. Furthermore, because the DWT is not a translation-invariant (TI) transform, results of processing depend on the relative alignment between data and wavelets in a complicated manner. In the context of denoising, this non-stationarity can produce undesirable ("pseudo-Gibbs" or "blocking") artifacts. To overcome these deficiencies, we propose a new wavelet-based signal denoising technique derived using the theory of non-informative Bayesian priors. The resulting estimator is TI and employs a very simple fixed non-linear shrinkage/thresholding rule. Remarkably, our new approach is very computationally efficient and performs better than standard methods that are more computationally demanding.
BibTeX
@misc{figueiredo2002bayesian,
  author = {Mário A. T. Figueiredo and Robert D. Nowak},
  title = {{Bayesian wavelet-based signal estimation using non-informative priors}},
  year = {2002},
  doi = {10.1109/acssc.1998.751550},
}

Loading related content...

Figure