Practical Compact Deep Compressed Sensing

Bin Chen, Jian Zhang

TL;DR

PCNet is a novel deep learning framework for Compressed Sensing (CS) that introduces a "collaborative sampling operator" (COSO) to bridge the gap between mathematical efficiency and neural adaptivity. It offers a scalable, hardware-ready solution that significantly outperforms state-of-the-art methods, particularly on high-resolution 4K images, while remaining interpretable and easy to deploy on physical devices like Digital Micro-mirror Devices (DMDs).

What Problem Does This Paper Address?

Compressed Sensing (CS) allows us to reconstruct high-resolution images from a fraction of the usual measurements, saving time, energy, and bandwidth. While deep learning has revolutionized CS reconstruction, the "sampling" part—how we actually capture those few measurements—remains a bottleneck.

Existing deep CS methods suffer from three main issues:

  1. Limited Receptive Fields: Most use block-based sampling, which ignores global image context and creates "blocking artifacts."
  2. Interpretability: Learned sampling operators are often "black boxes," making it unclear why they work or how to optimize them.
  3. Hardware Disconnect: Most neural sampling operators are difficult to translate into the explicit bitmask patterns required by real-world hardware like single-pixel cameras or DMDs.

Methods

The authors propose PCNet, which features two core innovations:

1. COSO (Collaborative Sampling Operator)

Instead of learning a single dense matrix, COSO uses a two-step process:

  • Deep Conditional Filtering: A lightweight 7-layer CNN adaptively smooths the input image based on the desired sampling rate.
  • Dual-branch Fast Sampling: The signal is split. One branch uses a Discrete Cosine Transform (DCT) to capture essential low-frequency structures. The other uses a scrambled block-diagonal Gaussian matrix, which provides a global receptive field at a fraction of the computational cost of a dense matrix.

2. Modernized Reconstruction Backbone

The reconstruction uses an unrolled Proximal Gradient Descent (PGD) algorithm. The authors "modernized" this classic optimization framework with ten specific enhancements, including the use of Swin-Conv blocks (combining Transformers and CNNs) and spatial downscaling to improve efficiency and recovery quality.

3. Deployment-Oriented Extraction

Crucially, the paper provides an algorithm to extract the explicit matrix form from their neural sampling operator. This allows researchers to take a trained model and generate the exact patterns needed to program optical hardware.

Datasets

The models were trained on:

  • Train400: A standard set of 400 natural images.
  • WED (Waterloo Exploration Database): A larger, higher-quality dataset of 4,744 images used to push the "PCT" (Transformer) version of the network.

Evaluation was performed on standard benchmarks (Set11, CBSD68, Urban100, DIV2K) and high-resolution sets (Test2K and Test4K) to demonstrate performance on modern image scales.

Key Insights and Ideas

  • Hybrid Sampling is King: Purely mathematical matrices (like Gaussian) are robust but "dumb," while purely learned matrices are adaptive but hard to scale. By combining a learned pre-filter with fixed transforms (DCT/Gaussian), PCNet gets the best of both worlds.
  • The Power of Scrambling: By simply shuffling pixels before block-sampling, the authors achieve a "global" receptive field. This prevents the network from becoming "blind" to large-scale structures without the $O(N^2)$ memory explosion of a global dense matrix.
  • Scalability via Conditioning: PCNet is "scalable," meaning a single set of weights can handle any sampling rate (e.g., from 1% to 50%). This is achieved by feeding the sampling rate as a "condition" into the filtering layers.
  • Modernizing the "Old": The paper demonstrates that "modernizing" a classic unrolled PGD architecture with recent CV tricks (better loss functions, larger datasets, and Transformer blocks) yields massive gains—up to 4dB over basic baselines.

Key Results

  • State-of-the-Art Accuracy: PCNet (and its Transformer variant, PCT) outperforms existing best methods by 0.37–1.88 dB on standard benchmarks.
  • High-Resolution Dominance: On 4K images, the performance gap widens significantly. PCNet leads other methods by 3–6 dB, proving that global sampling is essential for high-res data.
  • Quantization Robustness: In tests with 1-bit or 8-bit measurements (Quantized CS), PCNet maintained a significant lead, showing it preserves more information even when data is heavily compressed.
  • Self-Supervised Success: When used in frameworks that learn without ground-truth images, the PCNet sampling operator improved reconstruction by ~2 dB over traditional Gaussian sampling.

Limitations and Future Work

  • Computational Complexity: While the sampling is extremely fast, the reconstruction (especially the Transformer-based PCT) is computationally heavier than simple CNN-based decoders.
  • Physical Implementation: While the paper provides the math to extract matrices for DMD hardware, it does not show results from a physical optical bench setup. The authors note that applying this to real-world single-pixel imaging systems is the next logical step.
  • Domain Specificity: The current work focuses on natural images. Future work could explore how these "collaborative" operators perform in specialized fields like medical MRI or hyperspectral imaging.

Summary updated

Abstract

Recent years have witnessed the success of deep networks in compressed sensing (CS), which allows for a significant reduction in sampling cost and has gained growing attention since its inception. In this paper, we propose a new practical and compact network dubbed PCNet for general image CS. Specifically, in PCNet, a novel collaborative sampling operator is designed, which consists of a deep conditional filtering step and a dual-branch fast sampling step. The former learns an implicit representation of a linear transformation matrix into a few convolutions and first performs adaptive local filtering on the input image, while the latter then uses a discrete cosine transform and a scrambled block-diagonal Gaussian matrix to generate under-sampled measurements. Our PCNet is equipped with an enhanced proximal gradient descent algorithm-unrolled network for reconstruction. It offers flexibility, interpretability, and strong recovery performance for arbitrary sampling rates once trained. Additionally, we provide a deployment-oriented extraction scheme for single-pixel CS imaging systems, which allows for the convenient conversion of any linear sampling operator to its matrix form to be loaded onto hardware like digital micro-mirror devices. Extensive experiments on natural image CS, quantized CS, and self-supervised CS demonstrate the superior reconstruction accuracy and generalization ability of PCNet compared to existing state-of-the-art methods, particularly for high-resolution images. Code is available at https://github.com/Guaishou74851/PCNet.
BibTeX
@article{chen2025practical,
  author = {Bin Chen and Jian Zhang},
  title = {{Practical Compact Deep Compressed Sensing}},
  journal = {IEEE Trans. Pattern Anal. Mach. Intell.},
  year = {2025},
  doi = {10.1109/tpami.2024.3504490},
}
Figure