Learning to invert: Signal recovery via Deep Convolutional Networks

Ali Mousavi, Richard G. Baraniuk

Abstract

Compressive Sensing (CS) recovery traditionally involves solving complex iterative optimization problems, which limits real-time application. This paper introduces DeepInverse, a deep learning framework that replaces these iterative algorithms with a deep convolutional network (DCN). By shifting the computational burden to an off-line training phase, DeepInverse achieves signal reconstruction speeds hundreds of times faster than state-of-the-art methods like D-AMP and Total Variation (TV) minimization, while maintaining comparable accuracy and enhanced robustness at extremely low sampling rates.

Introduction

Compressive Sensing (CS) provides a framework for recovering high-dimensional signals from a limited number of linear measurements. Despite its theoretical elegance, CS faces two primary practical bottlenecks:

  1. Computational Complexity: Iterative recovery algorithms (e.g., $L_1$ minimization) are slow, hindering real-time processing for high-resolution data.
  2. Rigid Sparsity Models: Traditional CS assumes signals are sparse in hand-crafted bases (e.g., Wavelets). Real-world data often deviates from these models, leading to reconstruction artifacts.

DeepInverse addresses these challenges by "learning" the inverse mapping directly from data, effectively replacing mathematical assumptions with learned data-driven priors.

Methodology

The DeepInverse architecture is a specialized DCN designed to map a low-dimensional measurement vector $y$ back to a high-dimensional signal $x$.

The DeepInverse architecture: A measurement vector is transformed into a signal proxy via an adjoint operator, followed by refinement through convolutional layers.

Architectural Components

  • Dimensionality Boosting: The input $y$ is first processed by a fully connected linear layer representing the adjoint operator ($\Phi^\intercal$). This produces a "signal proxy" $\tilde{x}$ with the same dimensions as the target image.
  • Convolutional Refinement: Three convolutional layers follow the proxy generation. These layers are trained to remove aliasing artifacts inherent in undersampled measurements.
  • Design Choices: To maintain spatial resolution, the network avoids downsampling (max-pooling). Non-linearity is introduced via Rectified Linear Units (ReLU) after each convolutional layer.

The model was trained on the ImageNet dataset using $64 \times 64$ sub-image crops. Testing was conducted on a disjoint set of images to evaluate generalization performance.

Experimental Results

DeepInverse demonstrates significant improvements in reconstruction speed and competitive performance in signal quality.

Speed and Efficiency

The primary advantage of DeepInverse is its inference speed. While iterative methods like D-AMP and TV minimization require seconds per image (~3s and ~2.5s respectively), DeepInverse completes reconstruction in approximately 0.01 seconds. This ~300x speedup enables real-time signal recovery.

Reconstruction Quality

As shown in the performance analysis, DeepInverse performs comparably to state-of-the-art iterative algorithms across various undersampling ratios.

Average PSNR vs. Undersampling Ratio for DeepInverse compared to D-AMP, TV Minimization, and P-AMP.

Notably, at extremely low undersampling ratios (e.g., 0.01), DeepInverse significantly outperforms traditional methods. This suggests that the learned priors within the DCN are more robust than fixed sparsity assumptions when measurement data is extremely scarce. Furthermore, the network exhibits superior resilience to input noise at an undersampling ratio of 0.1 compared to D-AMP.

Discussion and Conclusion

DeepInverse represents a paradigm shift in compressive sensing by off-loading the computational complexity of signal recovery to an offline training phase. While the current 3-layer implementation shows promise, future work could explore deeper architectures to further improve quality. Current limitations include the requirement for a fixed measurement matrix ($\Phi$) and the need for domain-specific fine-tuning when moving between disparate signal classes. Nonetheless, the massive speedup and competitive accuracy make DeepInverse a viable candidate for real-time inverse problem applications.

Summary updated

BibTeX
@inproceedings{mousavi2017learning,
  author = {Ali Mousavi and Richard G. Baraniuk},
  title = {{Learning to invert: Signal recovery via Deep Convolutional Networks}},
  booktitle = {IEEE International Conference on Acoustics, Speech and Signal Processing},
  year = {2017},
  doi = {10.1109/icassp.2017.7952561},
}
Figure