Joint Image Denoising and Interpolation via Mixed Graph Filtering

Gene Cheung, Niruhan Viswarupan, Romisaa ElsaidyarXiv 2026 · arXiv

TL;DR

This paper introduces a unified mathematical framework for performing image denoising and interpolation simultaneously rather than sequentially. By modeling these tasks using a "mixed graph" approach—where denoising uses undirected edges and interpolation uses directed ones—the authors derive jointly optimal operators that significantly outperform traditional step-by-step pipelines.

What Problem Does This Paper Address?

In standard image processing, denoising and interpolation are often treated as independent, sequential steps. However, this "divide and conquer" strategy is fundamentally suboptimal:

  1. Interpolate then Denoise: Interpolation spreads local pixel noise to neighbors, creating correlated noise patterns that are much harder for standard denoisers to remove.
  2. Denoise then Interpolate: Denoising filters often smooth out fine textures and high-frequency details. Once these details are lost, the interpolator cannot use them to reconstruct a sharp upsampled image.

The authors tackle this "chicken-and-egg" problem by asking: Given an existing denoiser and interpolator, how can we combine them into a single, mathematically optimal joint operation?

Methods

The researchers leverage Graph Signal Processing (GSP) to bridge the gap between disparate filtering tasks. Their approach rests on several key technical pillars:

  • Graph-Regularized MAP Interpretations: They prove two foundational theorems showing that any (pseudo-)linear denoiser or interpolator can be viewed as a solution to a Maximum A Posteriori (MAP) estimation problem. Specifically, denoising corresponds to an undirected graph (where pixels influence each other symmetrically), while interpolation corresponds to a directed graph (where information flows one-way from observed to new pixels).
  • Mixed Graph Filtering: They propose a "mixed graph" model that incorporates both directed and undirected edges. This allows them to formulate a single optimization objective that accounts for noise in the original pixels, the interpolation mapping, and noise in the resulting interpolated pixels.
  • Separability Analysis: The paper characterizes exactly when a joint problem can be solved sequentially (separable) and when it requires a truly coupled solution (non-separable). They show that if you add a denoising prior to the interpolated pixels, the solution becomes non-separable, necessitating new joint operators ($\Theta^*$ and $\Psi^*$).
Fig. 1: Overview of proposed joint image denoising / interpo- lation framework. Given (pseudo-)linear denoisers Ψ and ¯Ψ for respective original and interpolated pixels and interpolator Θ, we first compute joint operators Θ∗and Ψ∗, then compute output image y = Θ∗Ψ∗x for input image x.
  • Efficient Implementation: To avoid the prohibitive cost of large matrix inversions ($O(N^3)$), the authors use CLIME (Constrained $\ell_1$-Minimization for Inverse Matrix Estimation) to find sparse approximations of graph operators. The final image is then reconstructed using a Conjugate Gradient (CG) solver.

Datasets

The framework was evaluated on several standard image processing benchmarks:

  • Stained-glass image (496 × 496): Used specifically for evaluating joint demosaicking and denoising.
  • Peppers image (512 × 512): Used for rotation and warping (homography) tasks.
  • Airplane image (512 × 512): Used to verify that performance trends were consistent across different image content. Evaluation was performed using Peak Signal-to-Noise Ratio (PSNR) across various levels of additive white Gaussian noise.

Key Insights and Ideas

  • Symmetry vs. Directionality: The most elegant insight is the mapping of tasks to graph types. Denoising is inherently about local similarity (undirected), while interpolation is about predictive flow (directed).
  • The "Non-Separable" Trigger: A joint solution only becomes "genuinely coupled" when we care about the smoothness of the newly interpolated pixels themselves. If we only denoise the source pixels, the optimal solution naturally falls back into a sequential pipeline.
  • Plug-and-Play Compatibility: Unlike deep learning methods that require retraining for every new task, this framework can "wrap" existing off-the-shelf denoisers (like Bilateral Filters or Non-Local Means) and interpolators, making it highly adaptable.
Fig. 2: Illustrations of mixed graphs for joint interpolation / denoising. Pixels 5,6,7,8 are interpolated using pixels 1,2,3,4. Directed edges for interpolation are shown as red dashed arrows, while undirected edges for denoising are shown as green dashed lines. (a) corresponds to corollary 1 where input pixels are denoised before interpolation. (b) corresponds to corollary 2 where input pixels are denoised before interpola- tion and interpolated pixels are denoised after interpolation.

Key Results

  • Demosaicking: The joint framework achieved a maximum PSNR gain of 1.9 dB over the sequential baseline at high noise levels ($\sigma^2 = 0.1$).
  • Rotation & Warping: For tasks like image rotation combined with Gaussian or Bilateral denoising, the joint approach consistently outperformed sequential pipelines by 0.4 dB to 0.7 dB.
  • Visual Quality: Qualitative results show that the joint method preserves significantly more detail in textured regions (e.g., the folds of a curtain) where sequential methods tend to over-smooth or leave "blotchy" noise artifacts.
  • Sparsity Trade-off: Using CLIME allowed the authors to increase the number of zeros in the system matrix from 35.5% to 53.3% with only a negligible (0.08 dB) loss in PSNR, proving the efficiency of the sparse approximation.

Limitations and Future Work

  • Linearity Assumption: The theoretical proofs rely on (pseudo-)linear operators. While many powerful filters (like Bilateral Filtering) can be treated as pseudo-linear, the framework might not directly extend to highly non-linear deep learning-based denoisers without further adaptation.
  • Computational Overhead: While the use of CG and CLIME makes the method "efficient," it is still computationally more demanding than a simple two-step spatial filter. The authors do not provide detailed real-time performance metrics or hardware acceleration strategies.
  • Future Directions: The authors suggest extending this mixed-graph approach to other multi-task problems, such as joint deblurring and super-resolution.

Summary updated

Abstract

Images often require both denoising and interpolation, yet treating the two tasks separately is suboptimal: interpolation introduces correlated noise that complicates subsequent denoising, while denoising may remove image details useful for interpolation. In this paper, we study joint image denoising and interpolation from a mixed graph filtering perspective, where denoising is modeled using undirected graph filters and interpolation using directed graph filters. We first establish that, under mild conditions, any (pseudo-)linear denoiser and interpolator can be interpreted as solution filters of maximum a posteriori (MAP) estimation problems regularized by undirected and directed graph smoothness priors, respectively. Guided by these results, we formulate graph-based joint denoising and interpolation problems using both directed and undirected graph structures. We characterize when joint denoising/interpolation reduces to a separable pipeline and when a genuinely coupled optimization is required. For the non-separable case, we recover sparse graph operators from given denoisers and interpolators, yielding an efficient linear-system formulation solvable via conjugate gradient. Experimental results demonstrate that our proposed joint framework consistently outperforms separate denoising and interpolation pipelines across multiple denoiser-interpolator combinations.
BibTeX
@misc{cheung2026joint,
  author = {Gene Cheung and Niruhan Viswarupan and Romisaa Elsaidy},
  title = {{Joint Image Denoising and Interpolation via Mixed Graph Filtering}},
  year = {2026},
  eprint = {2611.24242},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.24242},
}
Figure