TL;DR
ELDER (Explicit Learned Deep Equilibrium Regularizer) is a new framework that learns explicit mathematical functions to regularize imaging inverse problems. By combining Deep Equilibrium (DEQ) training with classical optimization theory, it achieves state-of-the-art reconstruction performance while providing the convergence guarantees that "black-box" neural networks often lack.
What Problem Does This Paper Address?
In computational imaging, we often try to solve "inverse problems"—recovering a clean image from data that is blurry, noisy, or incomplete (like a sparse MRI scan). Traditionally, this was done using hand-crafted mathematical "regularizers" that penalize unlikely images.
Recently, Deep Learning (DL) has taken over, but most DL methods use implicit regularization. In these models, a Convolutional Neural Network (CNN) acts as a denoiser within an iterative loop. While powerful, these models are hard to analyze; it’s difficult to prove they will always converge to a stable solution. On the other hand, explicit regularizers—where the CNN defines an actual energy function to be minimized—are theoretically safer but have historically struggled to match the performance of their implicit cousins.
Furthermore, training deep iterative models usually requires "unrolling" the iterations, which consumes massive amounts of GPU memory, limiting how "deep" or complex the model can be.
Methods
The authors introduce ELDER, which solves these issues using two key innovations:
- Deep Equilibrium (DEQ) Training: Instead of training a network by unrolling a fixed number of steps, ELDER treats the reconstruction as finding a "fixed point" (an equilibrium) of an algorithm. Using implicit differentiation, it can backpropagate through this equilibrium point with constant memory complexity, regardless of how many iterations are needed during inference.
- Explicit Parameterization: ELDER parameterizes the regularization function $h(x)$ using a CNN. The authors tested three specific architectures:
- LSR (Least-Squares Residual): Penalizes the distance between the image and the output of a CNN denoiser.
- RED (Regularization by Denoising): A functional based on the cross-product of the image and the denoiser residual.
- DSV (Direct Scalar-Valued): A network that directly outputs a single scalar "score" for an image.
By having an explicit objective function, the authors can use standard optimization tools like backtracking line-search to automatically select the best step size during reconstruction, ensuring the algorithm always moves toward a better solution.
Datasets
The models were trained and evaluated on several standard computer vision and medical imaging benchmarks:
- General Imaging: CBSD68, BSD68, and Kodak24 datasets were used for denoising, super-resolution, and inpainting tasks.
- Medical Imaging: A brain MRI dataset (800 training slices, 50 testing) was used for Compressed Sensing MRI (CS-MRI) evaluations.
- Training Data: A large composite dataset including CBSD, Waterloo Exploration Database, DIV2K, and Flickr2K.
Key Insights and Ideas
- Explicit is not a Compromise: The most significant "aha moment" is that you don't have to sacrifice performance to get theoretical stability. ELDER matches or beats implicit methods while remaining firmly rooted in optimization theory.
- LSR is the Strongest Prior: Among the three explicit forms tested, the Least-Squares Residual (LSR) parameterization consistently performed best, suggesting that "closeness to a denoised version" is a highly effective mathematical prior for deep learning.
- Stability Without Constraints: Most DEQ models for imaging require strict "Lipschitz constraints" (mathematical limits on how much the network can change its output) to stay stable. Because ELDER minimizes an explicit energy function, it inherits stability naturally from optimization theory without needing these restrictive constraints that often hamper performance.
Key Results
- Super-Resolution: ELDER significantly outperformed GSPnP (a recent explicit method) and matched the performance of state-of-the-art implicit methods like DPIR and standard DEQ.
- CS-MRI: At a 10% sampling ratio, ELDER achieved a PSNR of 35.14 dB, essentially matching the performance of the best implicit DEQ models (35.16 dB) while offering better convergence properties.
- Inpainting: ELDER showed a substantial lead over traditional Plug-and-Play (PnP) methods, recovering fine details and sharper edges in regions with 70% missing pixels.
- Memory Efficiency: Thanks to the DEQ approach, ELDER can be trained on a single GPU even for complex, high-iteration tasks that would crash standard "unrolled" models.
Limitations and Future Work
While ELDER is a major step forward, the authors note a few areas for growth:
- Jacobian-Free Approximation: To speed up training, the authors use a "Jacobian-free" approximation in the backward pass. While effective, exploring more exact or efficient ways to compute these gradients could further improve accuracy.
- Architectural Exploration: The study focused on the form of the regularizer (LSR vs RED vs DSV). Future work could explore how different CNN backbone architectures (like Transformers or newer U-Nets) interact with the ELDER framework.
- Non-Linear Problems: Currently, the focus is on linear inverse problems; extending this to non-linear forward models (like phase retrieval) is a logical next step.
Summary updated