Abstract: In this paper, we propose a new framework to remove parts of the systematic errors affecting popular restoration algorithms, with a special focus on image processing tasks. Generalizing ideas that emerged for $\ell_1$ regularization, we develop an approach refitting the results of standard methods toward the input data. Total variation regularizations and nonlocal means are special cases of interest. We identify important covariant information that should be preserved by the refitting method and emphasize the importance of preserving the Jacobian (w.r.t. the observed signal) of the original estimator. Then, we provide an approach that has a “twicing” flavor and allows refitting the restored signal by adding back a local affine transformation of the residual term. We illustrate the benefits of our method on numerical simulations for image restoration tasks.
Introduction
Popular image restoration algorithms like Lasso and Total Variation (TV) are widely used due to their ability to enforce structural properties like sparsity and sharp edges. However, these methods often suffer from a systematic "bias tax"—a tendency to shrink signal intensities and reduce contrast. This paper introduces CLEAR (Covariant LEAst-square Re-fitting), a framework designed to mitigate this bias while preserving the structural integrity of the original restoration.
The Bias Tax and the Need for Refitting
Methods relying on $\ell_1$ regularization, such as TV denoising, systematically push values toward zero. In 1D signals, this manifests as reduced step heights; in 2D images, it results in dimmed contrast and loss of fine details. Traditional "re-fitting" attempts, like the Hybrid Lasso, often struggle with stability because they depend on perfectly identifying the "support" (e.g., exact edge locations), which is difficult in the presence of noise.
The CLEAR Framework
CLEAR generalizes re-fitting by focusing on the Jacobian of the original estimator. The Jacobian captures the local sensitivity of the restored signal to the input data, effectively encoding the underlying model's constraints (e.g., smoothness or local interactions).
Key components of the method include:
- Model Subspace: CLEAR defines an affine subspace that captures the local linear invariance of the estimator.
- Covariant Preservation: Unlike "invariant re-fitting" which just projects data onto a subspace, CLEAR ensures the new estimator shares the same first-order behavior (the Jacobian) as the original.
- Twicing Flavor: The final algorithm adds a local affine transformation of the residual (the difference between the noisy input and the first estimate) back to the original estimate.
Experimental Evaluation
The authors tested CLEAR on synthetic phantoms and standard natural images (e.g., Cameraman, Lena) under various degradations like Gaussian noise, pixel masking, and blur. The framework was applied to several estimators, including anisotropic/isotropic TV, Non-Local Means (NLM), and BM3D.
Results and Visual Fidelity
CLEAR consistently recovers lost contrast and enhances thin structures that standard TV often "dims." For the Shepp-Logan phantom, CLEAR improved PSNR from ~30dB to over 37dB and the SSIM from 0.90 to 0.98. Visually, the method avoids the blurry look of over-smoothed regions in patch-based methods like NLM.
Unlike support-based refitting, CLEAR is continuous and robust, avoiding the "suspicious oscillations" and artifacts that occur when edge locations are slightly misidentified.
Conclusion
While CLEAR significantly reduces bias, it involves a trade-off by slightly increasing variance (noise). It is most effective when the original model is a good fit for the underlying signal. Despite the computational overhead of calculating the Jacobian—typically doubling or tripling processing time—CLEAR provides a principled, two-step procedure for high-fidelity image restoration.
Summary updated