Recursions Are All You Need: Towards Efficient Deep Unfolding Networks

Rawwad Alhejaili, Motaz Alfarraj, Hamzah Luqman, Ali Al-ShaikhiWorkshops @ CVPR 2023 · DOI · arXiv

Introduction

Compressive Sensing (CS) is a signal processing paradigm that enables the reconstruction of signals from a significantly reduced number of samples, far below the Nyquist rate. Deep Unfolding Networks (DUNs) have emerged as a leading solution in this domain, mapping traditional iterative optimization algorithms into neural network layers. While DUNs offer high performance and mathematical interpretability, they are often characterized by high parameter redundancy, as each iteration typically employs a unique set of learnable weights. This research introduces a recursive framework designed to enhance the efficiency of DUNs by replacing redundant layers with a single recursive unit and a novel feature modulation strategy.

Methodology

The proposed approach addresses the efficiency bottleneck through three primary innovations:

  1. Recursive Weight Sharing: Instead of utilizing $N$ distinct recovery blocks for $N$ iterations, the model employs a single recursive unit. This drastically reduces the total parameter count, acting as a form of natural regularization that improves generalization, particularly on smaller datasets.
  2. Recursion-based Feature Modulation Unit (RFMU): To address the varying feature requirements across different stages of reconstruction, the RFMU dynamically adjusts features based on the current iteration index ($R_{cur}$) and the total iterations ($R_{tot}$). This provides the necessary temporal context for the recursive block to adapt its behavior from early to late stages of recovery.
  3. Randomized Recursions during Training: During the training phase, the number of iterations per layer (IPL) is randomized. This strategy forces the network to learn robust reconstructions across varying step counts and significantly reduces training time by decreasing the average number of floating-point operations (FLOPs).

Experimental Results

The framework was evaluated using the ISTA-Net+ and COAST models on standard benchmarks such as BSD68 and Set11.

Efficiency and Performance

The recursive implementation achieved a parameter reduction of up to 75% for the COAST model and 66% for ISTA-Net+. Furthermore, training times were reduced by 21% to 42%. Despite these massive cuts in complexity, the models maintained nearly identical PSNR and SSIM scores compared to their non-recursive counterparts. In data-efficiency tests using only 3% of the training data, the recursive ISTA-Net+ actually outperformed the full baseline, demonstrating superior resilience to overfitting.

Visual Reconstruction Quality

The recursive process progressively refines the image quality through successive iterations. As shown in the comparison between the ground truth and the reconstructed output at the 7th iteration, the model successfully recovers fine details and structural integrity from compressed samples.

Ground truth image used for benchmarking.

Conclusion

By integrating recursive weight sharing with dynamic feature modulation, this work demonstrates that "recursions are all you need" to build efficient and high-performing deep unfolding networks. This framework not only slashes computational overhead but also enhances model robustness. Future research will explore performing these recursions in the latent feature space to further optimize representation learning and extend the framework to other restoration tasks like denoising and super-resolution.

Summary updated

BibTeX
@inproceedings{alhejaili2023recursions,
  author = {Rawwad Alhejaili and Motaz Alfarraj and Hamzah Luqman and Ali Al-Shaikhi},
  title = {{Recursions Are All You Need: Towards Efficient Deep Unfolding Networks}},
  booktitle = {Workshops @ CVPR},
  year = {2023},
  doi = {10.1109/cvprw59228.2023.00498},
}
Figure