TL;DR
This paper introduces a novel, computationally efficient, and translation-invariant (TI) wavelet-based signal denoising method. By leveraging non-informative Bayesian priors, it achieves state-of-the-art performance without the need for arbitrary parameter tuning, overcoming limitations of existing methods.
What Problem Does This Paper Address?
Signal denoising is a crucial task in many fields, and the Discrete Wavelet Transform (DWT) has proven highly effective due to its ability to represent signals sparsely and decorrelate them. However, existing DWT-based denoising methods often suffer from:
- Arbitrary Thresholding: Many methods rely on ad hoc threshold levels or non-linear shrinkage functions that require manual tuning for optimal results.
- Computational Expense: Signal-adaptive schemes, while potentially more accurate, can be computationally demanding.
- Translation Variance: The DWT is not translation-invariant, meaning that shifting the input signal can lead to significantly different denoising results, often introducing undesirable artifacts like "pseudo-Gibbs" or "blocking."
While translation-invariant (TI) denoising approaches exist (typically by averaging results from multiple shifts), they often lack a strong theoretical foundation. This paper aims to address these issues by developing a principled, Bayesian approach to TI denoising that is both theoretically sound and practically efficient.
Methods
The authors propose a new wavelet-based signal denoising technique grounded in Bayesian inference and the theory of non-informative priors. Key aspects of their approach include:
- Bayesian Formulation: The denoising problem is framed within a Bayesian context, where the unknown signal is modeled with a prior distribution, and observations are modeled with a likelihood function. Bayes' theorem is used to derive the posterior distribution, from which an optimal estimate can be obtained based on a chosen loss function.
- Non-Informative Priors: To mitigate the arbitrariness of existing methods, the authors introduce a novel non-informative prior for the wavelet coefficients. This prior is designed to be scale-invariant and avoids introducing free parameters that require tuning. Specifically, they use a hierarchical Bayesian model with a Jeffreys prior on the variance of the wavelet coefficients.
- Translation-Invariant (TI) Denoising: The paper derives a formal Bayesian criterion for TI denoising. This involves treating the signal shift as an unknown and using a uniform (non-informative) prior over all possible shifts. The resulting TI estimator is shown to be a weighted average of shift-dependent estimators, where the weights are determined by the evidence supporting each shift.
- Empirical Bayes Approach: To handle the improper posterior distribution arising from the non-informative prior, the authors adopt an empirical Bayes approach. This involves estimating certain parameters (like the variance of the wavelet coefficients) from the data, effectively breaking the fully Bayesian chain.
- Novel Shrinkage/Thresholding Rule: The proposed Bayesian framework leads to a simple, fixed non-linear shrinkage/thresholding rule for individual wavelet coefficients. This rule is shown to behave like soft thresholding for small coefficients and hard thresholding for large ones, offering a balance between the two classical approaches.
Datasets
The paper evaluates its proposed method using standard test signals commonly employed in signal processing research for denoising benchmarks. These include:
- Blocks
- Doppler
- HeaviSine
- Bumps
These signals are used to compare the performance of the new algorithm against existing methods across various Signal-to-Noise Ratios (SNRs). No new datasets or benchmarks are introduced.
Key Insights and Ideas
- Principled TI Denoising: The paper provides the first formal estimation-theoretic derivation of translation-invariant (TI) denoising within a Bayesian framework, moving beyond ad hoc averaging methods.
- Non-Informative Priors for Simplicity and Performance: By using non-informative priors, the authors achieve a denoising rule with no free parameters to tune, making it easy to apply and computationally efficient, while still outperforming more complex adaptive methods.
- A Balanced Shrinkage Rule: The derived shrinkage/thresholding rule offers a compelling compromise between the aggressive shrinkage of soft thresholding and the more abrupt cutoff of hard thresholding, leading to improved denoising performance.
- Scale Invariance is Key: The emphasis on scale-invariant priors is highlighted as a desirable property that ensures the denoising results are robust to changes in measurement units or signal scaling.
Key Results
- Superior Performance: The proposed Bayesian wavelet-based denoising method consistently outperforms VisuShrink and SureShrink across various test signals and SNRs.
- Competitive with State-of-the-Art: The new method achieves performance comparable to, and in some cases better than, computationally more demanding empirical Bayesian methods (e.g., those using Gaussian mixtures).
- Computational Efficiency: Despite its advanced theoretical underpinnings, the proposed method is as computationally inexpensive as simpler schemes like VisuShrink.
- Reduced Artifacts: The TI nature of the proposed estimator, as demonstrated in Figure 4, leads to a visually cleaner denoised signal with fewer artifacts compared to non-TI methods.
Limitations and Future Work
- Empirical Bayes vs. Full Bayes: The authors acknowledge that their empirical Bayes approach breaks the fully Bayesian inference chain. While this simplifies computation, it means the derived TI estimator is not strictly optimal in a full Bayesian sense.
- Estimating Shift Probabilities: For the theoretically optimal TI estimator (Equation 14), calculating the evidence $p(s|y)$ (the weights for averaging shift-dependent estimates) is challenging, especially within an empirical Bayes framework. The authors state they are currently investigating methods for estimating these probabilities.
- Known Noise Variance: The formulation assumes the noise variance $\sigma^2$ is known. While they mention that good estimates can be obtained (e.g., from the MAD scheme), this is a common assumption in many denoising methods.
Future work will likely focus on developing robust methods for estimating the shift probabilities $p(s|y)$ to achieve the fully Bayesian optimal TI estimator and exploring the application of these techniques to other signal processing tasks.
Summary updated