CLEAR: Covariant LEAst-Square Refitting with Applications to Image Restoration

Charles-Alban Deledalle, Nicolas Papadakis, Joseph Salmon, Samuel Vaiter

Abstract: In this paper, we propose a new framework to remove parts of the systematic errors affecting popular restoration algorithms, with a special focus on image processing tasks. Generalizing ideas that emerged for $\ell_1$ regularization, we develop an approach refitting the results of standard methods toward the input data. Total variation regularizations and nonlocal means are special cases of interest. We identify important covariant information that should be preserved by the refitting method and emphasize the importance of preserving the Jacobian (w.r.t. the observed signal) of the original estimator. Then, we provide an approach that has a “twicing” flavor and allows refitting the restored signal by adding back a local affine transformation of the residual term. We illustrate the benefits of our method on numerical simulations for image restoration tasks.

Introduction

Popular image restoration algorithms like Lasso and Total Variation (TV) are widely used due to their ability to enforce structural properties like sparsity and sharp edges. However, these methods often suffer from a systematic "bias tax"—a tendency to shrink signal intensities and reduce contrast. This paper introduces CLEAR (Covariant LEAst-square Re-fitting), a framework designed to mitigate this bias while preserving the structural integrity of the original restoration.

The Bias Tax and the Need for Refitting

Methods relying on $\ell_1$ regularization, such as TV denoising, systematically push values toward zero. In 1D signals, this manifests as reduced step heights; in 2D images, it results in dimmed contrast and loss of fine details. Traditional "re-fitting" attempts, like the Hybrid Lasso, often struggle with stability because they depend on perfectly identifying the "support" (e.g., exact edge locations), which is difficult in the presence of noise.

1D signal denoising illustrating the bias of Total Variation (TV) and the geometric interpretation of the CLEAR refitting manifold.

The CLEAR Framework

CLEAR generalizes re-fitting by focusing on the Jacobian of the original estimator. The Jacobian captures the local sensitivity of the restored signal to the input data, effectively encoding the underlying model's constraints (e.g., smoothness or local interactions).

Key components of the method include:

  • Model Subspace: CLEAR defines an affine subspace that captures the local linear invariance of the estimator.
  • Covariant Preservation: Unlike "invariant re-fitting" which just projects data onto a subspace, CLEAR ensures the new estimator shares the same first-order behavior (the Jacobian) as the original.
  • Twicing Flavor: The final algorithm adds a local affine transformation of the residual (the difference between the noisy input and the first estimate) back to the original estimate.

Experimental Evaluation

The authors tested CLEAR on synthetic phantoms and standard natural images (e.g., Cameraman, Lena) under various degradations like Gaussian noise, pixel masking, and blur. The framework was applied to several estimators, including anisotropic/isotropic TV, Non-Local Means (NLM), and BM3D.

Results and Visual Fidelity

CLEAR consistently recovers lost contrast and enhances thin structures that standard TV often "dims." For the Shepp-Logan phantom, CLEAR improved PSNR from ~30dB to over 37dB and the SSIM from 0.90 to 0.98. Visually, the method avoids the blurry look of over-smoothed regions in patch-based methods like NLM.

Comparison of image restoration using anisotropic and isotropic TV with various refitting methods, showing CLEAR's ability to restore contrast and detail.

Unlike support-based refitting, CLEAR is continuous and robust, avoiding the "suspicious oscillations" and artifacts that occur when edge locations are slightly misidentified.

Conclusion

While CLEAR significantly reduces bias, it involves a trade-off by slightly increasing variance (noise). It is most effective when the original model is a good fit for the underlying signal. Despite the computational overhead of calculating the Jacobian—typically doubling or tripling processing time—CLEAR provides a principled, two-step procedure for high-fidelity image restoration.

Summary updated

Abstract

In this paper, we propose a new framework to remove parts of the systematic errors affecting popular restoration algorithms, with a special focus on image processing tasks. Generalizing ideas that emerged for $\ell_1$ regularization, we develop an approach refitting the results of standard methods toward the input data. Total variation regularizations and nonlocal means are special cases of interest. We identify important covariant information that should be preserved by the refitting method and emphasize the importance of preserving the Jacobian (w.r.t. the observed signal) of the original estimator. Then, we provide an approach that has a “twicing” flavor and allows refitting the restored signal by adding back a local affine transformation of the residual term. We illustrate the benefits of our method on numerical simulations for image restoration tasks.
BibTeX
@article{deledalle2017clear,
  author = {Charles-Alban Deledalle and Nicolas Papadakis and Joseph Salmon and Samuel Vaiter},
  title = {{CLEAR: Covariant LEAst-Square Refitting with Applications to Image Restoration}},
  journal = {SIAM J. Imaging Sciences},
  year = {2017},
  url = {https://openreview.net/forum?id=9SGaRHaI2r},
}
Figure