Abstract
We propose an adaptive learning procedure to learn patch-based image priors for image denoising. The new algorithm, called the Expectation-Maximization (EM) adaptation, takes a generic prior learned from a generic external database and adapts it to the noisy image to generate a specific prior. Different from existing methods that combine internal and external statistics in ad-hoc ways, the proposed algorithm is rigorously derived from a Bayesian hyper-prior perspective. There are two contributions of this paper: First, we provide full derivation of the EM adaptation algorithm and demonstrate methods to improve the computational complexity. Second, in the absence of the latent clean image, we show how EM adaptation can be modified based on pre-filtering. Experimental results show that the proposed adaptation algorithm yields consistently better denoising results than the one without adaptation and is superior to several state-of-the-art algorithms.
Introduction
In image denoising, the challenge lies in recovering a clean image $x$ from a noisy observation $y$. Modern techniques typically rely on image priors, which fall into two categories: external priors learned from large databases and internal priors learned from the noisy image itself. External priors are robust but generic, often missing image-specific textures. Internal priors are specific but prone to failure due to noise corruption and limited data.
This paper introduces EM adaptation, a principled Bayesian framework that bridges this gap. By treating a general Gaussian Mixture Model (GMM) as a "hyper-prior," the algorithm adapts to the unique features of a noisy image, consistently outperforming standard denoising benchmarks like EPLL and BM3D.
Methods
Bayesian Hyper-priors and Adaptation
Instead of treating GMM parameters as fixed, the authors treat mixture weights, means, and covariances as random variables. Using conjugate priors—specifically the Dirichlet distribution for weights and the Normal-Inverse-Wishart (NIW) distribution for means and covariances—the model is "nudged" toward specific image data while remaining anchored to the generic external model.
The resulting M-step updates are intuitive: adapted parameters are a weighted linear combination of the generic prior and new image data. For example, the adapted mean $\tilde{\mu}_k$ is calculated as: $$\tilde{\mu}_k = \alpha_k (\text{new data average}) + (1 - \alpha_k) (\text{generic mean})$$ where $\alpha_k$ is a data-dependent weight determining the trust level between the new image and the external database.
Noise Compensation and Efficiency
To handle noisy inputs, the E-step is modified to account for noise variance, utilizing Monte-Carlo SURE (Stein's Unbiased Risk Estimator) to estimate residual noise after pre-filtering. Furthermore, the authors derived a simplified covariance update that reduces complexity from $O(nd^2)$ to $O(d^2)$, enabling significant practical speedups.
Experimental Results
The method was evaluated using 2,000,000 patches for generic training and tested on standard benchmarks (Lena, House, Peppers) and specialized datasets like the 102 Flowers and FEI Face datasets.
Quantitative and Visual Gains
The adapted GMM (aGMM-EPLL) consistently outperformed standard EPLL by an average of 0.3 dB across various noise levels. Visually, the method excels at preserving sharp edges and fine textures, which is particularly evident in text and facial feature denoising.
Computational Performance
One of the most significant practical contributions is the optimization of the adaptation process. For a 256x256 image, adaptation takes approximately 14 seconds in MATLAB. As shown in the table below, the optimized derivation provides a massive speedup over the original formulation.
Conclusion
EM adaptation provides a principled way to fuse external and internal image statistics. While it relies on the quality of pre-filtered "pilot" images at very high noise levels ($\sigma > 60$), it offers a robust and efficient improvement to the standard MAP denoising pipeline. Future work includes extending this framework to video denoising, where temporal consistency offers additional data for adaptation.
Summary updated