Abstract
A noise-corrupted image often requires interpolation. Given a linear denoiser and a linear interpolator, when should the operations be independently executed in separate steps, and when should they be combined and jointly optimized? We study joint denoising / interpolation of images from a mixed graph filtering perspective: we model denoising using an undirected graph, and interpolation using a directed graph. We first prove that, under mild conditions, a linear denoiser is a solution graph filter to a maximum a posteriori (MAP) problem regularized using an undirected graph smoothness prior, while a linear interpolator is a solution to a MAP problem regularized using a directed graph smoothness prior. Next, we study two variants of the joint interpolation / denoising problem: a graph-based denoiser followed by an interpolator has an optimal separable solution, while an interpolator followed by a denoiser has an optimal non-separable solution. Experiments show that our joint denoising / interpolation method outperformed separate approaches noticeably.
Introduction
In modern image processing pipelines, denoising (removing sensor noise) and interpolation (resizing, rotating, or warping) are typically treated as independent, sequential modules. However, these operations are fundamentally coupled: noise in the source pixels propagates through the interpolation process, and interpolation itself alters the statistical structure of the noise. This paper introduces a unified mathematical framework using Graph Signal Processing (GSP) to determine when these operations can be performed independently and when they must be jointly optimized for maximum fidelity.
Graph-Based Modeling of Image Operations
The researchers leverage GSP to model the two tasks by viewing pixels as nodes on a graph. The directionality of the edges in these graphs dictates the flow of information and the optimal processing strategy.
- Undirected Graphs for Denoising: Denoising is modeled using undirected graphs where edges represent mutual similarity between pixels. The authors prove that standard linear denoisers, such as Gaussian or Bilateral filters, are equivalent to a Maximum A Posteriori (MAP) optimization problem using an undirected graph smoothness prior.
- Directed Graphs for Interpolation: Interpolation is modeled using directed graphs where "known" source pixels influence "new" target pixels, but not vice versa. This creates a one-way information flow that can also be framed as a MAP problem with a directed graph prior.
The Separability Theorem
The core contribution of the paper is the analysis of two distinct processing sequences:
- Denoising then Interpolating (Separable): When denoising occurs on the source grid before interpolation, the operations are mathematically separable. The optimal solution is simply to apply the denoiser and then the interpolator sequentially.
- Interpolating then Denoising (Non-separable): When interpolation is performed first, the noise is mapped onto the target grid. Denoising on this target grid requires accounting for the interpolation geometry. This coupling makes the problem non-separable. To solve this, the authors derive joint operators ($\Psi^*$ and $\Theta^*$) that combine both filters into a single optimization step.
Experimental Results
The proposed joint method was evaluated against standard sequential pipelines using classic benchmark images (Lena and peppers). The tasks included image rotation and homography-based warping across varying levels of additive Gaussian noise.
As shown in the results, the "Joint" approach consistently outperformed the "Sequential" approach. The performance gap widened as noise levels increased, demonstrating the robustness of the joint optimization. Specifically, for image rotation using a Bilateral denoiser, the joint method achieved a 1.35 dB PSNR gain. For image warping with a Gaussian denoiser, the gain was 1.14 dB.
Conclusion and Future Work
This research proves that the common practice of interpolating noisy data and then applying a standard denoiser is mathematically sub-optimal. By bridging traditional image processing and GSP, the authors provide a rigorous framework for joint optimization. While the current work focuses on linear filters, future research may extend these principles to non-linear deep learning-based denoisers or other mixed problems like joint deblurring and super-resolution.
Summary updated