Fast Compressive Sensing Recovery Using Generative Models with Structured Latent Variables

Shaojie Xu, Sihan Zeng, Justin Romberg

TL;DR

This paper introduces a novel algorithm for reconstructing high-dimensional signals from a limited number of compressed measurements. By leveraging Generative Adversarial Networks (GANs) with structured latent variables and an Alternating Direction Method of Multipliers (ADMM) framework, the method achieves significantly faster and more stable recovery compared to existing approaches, especially under extreme compression.

What Problem Does This Paper Address?

Compressive Sensing (CS) aims to reconstruct a high-dimensional signal (like an image) from far fewer measurements than traditionally required by Nyquist-Shannon sampling. This is possible because many real-world signals have inherent structure (e.g., sparsity). However, recovering these signals accurately, especially when the number of measurements is extremely limited, remains a significant challenge. Traditional CS methods often rely on assumptions like sparsity, which may not fully capture the complex structures of natural signals. More recent approaches use deep learning, particularly generative models like GANs, to learn these complex priors. While these generative models can capture rich signal statistics, their integration into CS recovery algorithms has often been slow and computationally expensive, typically relying on gradient descent (GD) for optimization. This paper addresses the need for a CS recovery method that is both fast and accurate, capable of exploiting powerful generative model priors even under severe compression.

Methods

The core of this paper is a novel algorithm that combines the strengths of Generative Adversarial Networks (GANs) for capturing signal priors with the efficiency of Alternating Direction Method of Multipliers (ADMM) for optimization.

  • Generative Model as Prior: A GAN is trained to learn the underlying data distribution, effectively capturing complex image structures. The latent space of the GAN is crucial.
  • Structured Latent Variables: The paper emphasizes the importance of a structured latent space. They propose using an InfoGAN setup, which disentangles the latent space into a "codeword" (controlling semantic meaning) and a "random-noise-like" variable (adding minor variations). This structured latent space is key to achieving stable recovery, especially when measurements are scarce.
  • ADMM-based Optimization: Instead of relying on computationally expensive gradient descent for recovery, the authors formulate the problem as an optimization problem and solve it using an ADMM-like framework. This involves alternating updates between the signal space and the latent variable space.
  • Projector Network: To efficiently search the latent variable space, they introduce a "projector neural network" (Gproj). This network is trained to map noisy versions of generated signals back to their corresponding latent variables. This replaces the slow GD step in the z-update.

The algorithm's update steps involve solving a least-squares problem for the signal x and using the projector network for the latent variable z. This ADMM-based approach with a learned projector allows for fast computation.

Datasets

  • MNIST: Used for training generative models and evaluating digit reconstruction.
  • CelebA: Used for training generative models and evaluating face image reconstruction.

The paper does not introduce new benchmarks but evaluates its method against existing ones on these standard datasets.

Key Insights and Ideas

  • Structured Latent Space is Crucial for Stability: The paper strongly argues that the structure of the GAN's latent space is not just for better prior representation but is critical for stable and accurate CS recovery, particularly under extreme compression. InfoGAN's disentangled latent space proves particularly effective.
  • ADMM + Projector Network = Fast Recovery: Combining ADMM with a learned projector network is a highly effective strategy for accelerating CS recovery. This avoids the slow gradient descent steps typically used with generative models.
  • Generative Models as "Plug-and-Play" Denoisers: The ADMM framework naturally lends itself to a "plug-and-play" interpretation, where the latent variable update acts like a sophisticated denoiser, leveraging the learned prior from the GAN.
  • Theoretical Guarantees: The paper provides theoretical recovery guarantees, showing how the number of measurements required scales with the dimension of the structured latent variables, offering a principled understanding of the method's performance.

Key Results

  • Improved Accuracy: The proposed algorithm, especially when using InfoGAN, achieves comparable or better reconstruction accuracy than gradient-descent-based GAN methods and other baselines, particularly at high compression ratios (e.g., 32x and 64x). For MNIST, it achieved 99.3% classification accuracy on reconstructed digits.
  • Significant Speedup: The ADMM-based approach with a projector network results in a substantial speedup. The paper reports a 20x speedup compared to gradient-descent-based GAN methods.
  • Stability under High Compression: The method demonstrates remarkable stability and produces high-quality images even when the number of compressed measurements is extremely limited, outperforming baselines that degrade significantly.
  • Preservation of Non-Smooth Features: The generative model prior allows the algorithm to preserve realistic and non-smooth features in the reconstructed images, which is often a challenge for methods relying on smoothness assumptions.

Limitations and Future Work

  • Training Complexity: While recovery is fast, training the generative model and the projector network can still be computationally intensive.
  • Generalization to Other Priors: The paper focuses on GANs and InfoGANs. Future work could explore other types of generative models or structured priors.
  • Theoretical Bounds: While theoretical guarantees are provided, they are based on specific assumptions about the generative model and the measurement matrix. Extending these bounds to more general scenarios could be valuable.
  • Real-world Applications: The authors suggest the method can be generalized to other inverse imaging problems beyond CS recovery, implying future work in areas like medical imaging or remote sensing.

Summary updated

Abstract

Deep learning models have significantly improved the visual quality and accuracy on compressive sensing recovery. In this paper, we propose an algorithm for signal reconstruction from compressed measurements with image priors captured by a generative model. We search and constrain on latent variable space to make the method stable when the number of compressed measurements is extremely limited. We show that, by exploiting certain structures of the latent variables, the proposed method produces improved reconstruction accuracy and preserves realistic and non-smooth features in the image. Our algorithm achieves high computation speed by projecting between the original signal space and the latent variable space in an alternating fashion.
BibTeX
@inproceedings{xu2019fast,
  author = {Shaojie Xu and Sihan Zeng and Justin Romberg},
  title = {{Fast Compressive Sensing Recovery Using Generative Models with Structured Latent Variables}},
  booktitle = {IEEE International Conference on Acoustics, Speech and Signal Processing},
  year = {2019},
  doi = {10.1109/icassp.2019.8683641},
}
Figure