TL;DR
This paper introduces the Fully Learnable Compressed Sensing (FLCS) model, a novel deep learning approach that learns a task-specific measurement matrix and reconstruction pipeline for efficient image acquisition at low sampling rates. FLCS significantly outperforms existing methods in image quality and reconstruction speed, making it highly relevant for resource-constrained applications.
What Problem Does This Paper Address?
Traditional Compressed Sensing (CS) relies on fixed, often random, measurement matrices and assumes signal sparsity in a predefined basis. While effective for general signals, this approach struggles with natural images at very low sampling rates, leading to poor reconstruction quality. This is because natural images are complex, and finding a universally optimal sparse representation is challenging. Furthermore, existing methods often require many iterations for reconstruction. This paper addresses the need for more efficient and higher-quality image reconstruction in CS, particularly for specific applications where a fixed, generic approach is suboptimal. They aim to overcome the limitations of traditional CS by learning a "task-driven" measurement process tailored to the type of images being acquired.
Methods
The core of this work is the Fully Learnable Compressed Sensing (FLCS) model, which integrates the sampling and reconstruction processes into a single, end-to-end trainable framework inspired by Generative Adversarial Networks (GANs) and Variational Autoencoders (VAEs). FLCS is divided into three key learnable components:
- Sampler (Sp): This module learns a task-specific measurement matrix ($\Phi$). Instead of using a fixed random matrix, FLCS learns a linear projection that is optimized for the target image domain. This is analogous to the encoder part of a VAE, which learns a low-dimensional representation.
- Solver (Sv): This component takes the measurements ($y$) and learns a nonlinear mapping to a low-dimensional latent representation ($z$). This is also derived from the VAE encoder, aiming to find the most appropriate latent representation of the original image.
- Rebuilder (R): This module reconstructs the image from the latent representation ($z$). It combines elements of a VAE decoder and a GAN generator, learning a nonlinear mapping from the latent space back to the image space.
These three modules can be trained jointly or individually. The entire FLCS model is trained using a combination of VAE and GAN loss functions, aiming to reconstruct high-quality images while fooling a discriminator. The reconstruction process then uses the pre-trained Sampler, Solver, and Rebuilder, requiring only a few iterations to refine the latent representation and produce the final image.
Datasets
The paper uses two publicly available datasets for training and evaluation:
- Aberystwyth leaf evaluation dataset: This dataset consists of images of Arabidopsis Thaliana plants grown in trays, captured over time. It's relevant for applications like crop monitoring.
- CelebA dataset: A large-scale dataset of celebrity faces, commonly used for face detection and recognition tasks.
The datasets were split into training (19,200 images) and testing (1000 images) sets. Images were preprocessed to a size of 64x64 pixels.
Key Insights and Ideas
- Task-Driven Measurement Matrix: The most significant insight is that a learned, task-specific measurement matrix can dramatically improve reconstruction quality compared to generic random matrices, especially at low sampling rates. This "learnable sensing" adapts to the inherent structure of the target image domain.
- End-to-End Learning for CS: Integrating sampling and reconstruction into a single, trainable deep learning framework allows for joint optimization, leading to better overall performance than treating these as separate problems.
- VAE-GAN Synergy: The combination of VAE and GAN architectures provides a powerful generative model capable of learning complex image manifolds and producing realistic reconstructions. The VAE component aids in finding a structured latent representation, while the GAN component ensures visual fidelity.
- Reduced Iterations for Reconstruction: The pre-trained FLCS model requires significantly fewer iterations for reconstruction compared to traditional iterative methods, leading to faster processing times.
Key Results
- Improved Image Quality: FLCS consistently outperformed state-of-the-art methods (TVAL3, NLRCS, DAMP, GAPTV, CSGM) in terms of Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity (SSIM) across various sampling rates (from 0.1 down to 0.01). For instance, at a sampling rate of 0.01, FLCS achieved PSNR and SSIM values significantly higher than other methods, particularly when those methods began to fail.
- Learned Matrix Advantage: FLCS-S (using a learned measurement matrix) showed superior performance over FLCS-G (using a Gaussian random matrix), with improvements of up to 4.8 dB in PSNR and 0.23 in SSIM on plant images at the lowest sampling rate.
- Faster Reconstruction: FLCS demonstrated significantly reduced reconstruction times. On average, it took 1.60s (CPU) / 0.14s (GPU) for FLCS to reconstruct an image at a sampling rate of 0.1, compared to 2.30s (CPU) / 0.25s (GPU) for CSGM. The runtime of FLCS remained relatively constant even as the sampling rate decreased.
- L1-norm Preference: Pre-training FLCS with an L1-norm loss function generally yielded better results than L2-norm, particularly in preserving image details and color accuracy.
Limitations and Future Work
- Training Time: While reconstruction is fast, the initial training of the FLCS model (including Sampler and Solver) is time-consuming, taking up to 8.74 hours on their experimental platform.
- Image Size: The current FLCS model is demonstrated on small image sizes (64x64).
- Future Work: The authors suggest exploring diverse generative model architectures, incorporating low-rank constraints, and increasing the training dataset size to further improve performance. They also plan to investigate post-processing techniques for handling abnormal image acquisitions and extending FLCS to reconstruct large-scale images by combining it with block-based CS methods.
Summary updated