Deep Network for Image Compressed Sensing Coding Using Local Structural Sampling

Wenxue Cui, Xingtao Wang, Xiaopeng Fan, Shaohui Liu, Xinwei Gao, Debin Zhao

Abstract

Existing image compressed sensing (CS) coding frameworks usually solve an inverse problem based on measurement coding and optimization-based image reconstruction, which still exist the following two challenges: (1) the widely used random sampling matrix, such as the Gaussian Random Matrix (GRM), usually leads to low measurement coding efficiency, and (2) the optimization-based reconstruction methods generally maintain a much higher computational complexity. In this article, we propose a new convolutional neural network based image CS coding framework using local structural sampling (dubbed CSCNet) that includes three functional modules: local structural sampling, measurement coding, and Laplacian pyramid reconstruction. In the proposed framework, instead of GRM, a new local structural sampling matrix is first developed, which is able to enhance the correlation between the measurements through a local perceptual sampling strategy. Besides, the designed local structural sampling matrix can be jointly optimized with the other functional modules during the training process. After sampling, the measurements with high correlations are produced, which are then coded into final bitstreams by the third-party image codec. Last, a Laplacian pyramid reconstruction network is proposed to efficiently recover the target image from the measurement domain to the image domain. Extensive experimental results demonstrate that the proposed scheme outperforms the existing state-of-the-art CS coding methods while maintaining fast computational speed.

Introduction

Compressed Sensing (CS) has revolutionized signal acquisition by enabling the capture of signals at rates significantly lower than the Nyquist-Shannon sampling theorem suggests. In the context of image processing, CS offers a path toward efficient compressive imaging. However, bridging the gap between theoretical CS and practical image coding remains challenging due to two primary hurdles.

First, most CS frameworks utilize random sampling matrices, such as the Gaussian Random Matrix (GRM). While mathematically sound, these matrices produce measurements that resemble noise, lacking the spatial correlation required for efficient compression by standard image codecs. Second, traditional reconstruction methods rely on iterative optimization algorithms that are computationally intensive and slow, making them unsuitable for real-time applications.

To address these issues, we introduce CSCNet, an end-to-end deep learning framework that integrates local structural sampling with a multi-scale reconstruction network to achieve high-performance image CS coding.

The CSCNet Framework

The CSCNet architecture is designed to be "codec-friendly" and computationally efficient. It comprises three primary functional modules: Local Structural Sampling (LSS), Measurement Coding, and Laplacian Pyramid Reconstruction.

The proposed CSCNet framework, illustrating the end-to-end pipeline from image sampling to reconstruction through a communication channel.

Local Structural Sampling (LSS)

The core innovation of CSCNet is the replacement of traditional random sampling with a learnable, sparse sampling matrix. Implemented as a convolutional layer, the LSS module is trained jointly with the reconstruction network. It incorporates two critical constraints:

  • Localization (LO): This ensures that the sampling matrix only perceives local image blocks. By restricting the receptive field, the module maintains the spatial correlation inherent in natural images within the measurement domain.
  • Positive Normalization (PNO): This constrains the range of the measurements to match the intensity levels of natural images, facilitating their processing by standard image codecs.

Measurement Coding

Unlike traditional CS measurements that appear as 1D noise-like vectors, the measurements produced by the LSS module are reshaped into 2D tensors. Because these "measurement images" preserve local structural correlations, they can be efficiently compressed using off-the-shelf third-party codecs like JPEG2000 or BPG (HEVC-Intra). This allows CSCNet to leverage existing, highly optimized image compression standards.

Laplacian Pyramid Reconstruction Network

To recover the original image from the compressed measurements, CSCNet employs a multi-scale Laplacian pyramid reconstruction network. This architecture progressively reconstructs the target image across multiple levels, from coarse to fine scales.

Detailed architecture of the Laplacian pyramid reconstruction network, showing the multi-level feature extraction and upsampling process.

As shown in the figure above, the network takes the decoded measurements and applies a series of convolutional layers and residual blocks at each level of the pyramid. This multi-scale approach allows the model to aggregate features efficiently, capturing both global structure and fine-grained details while maintaining a fast feed-forward execution.

Experimental Evaluation

The CSCNet model was trained using the BSDS500 dataset and the training set of VOC2012. Evaluation was performed using standard benchmark images (e.g., Lenna, Boats, Monarch) and the Set11 dataset.

Performance and Visual Quality

Experimental results indicate that CSCNet consistently outperforms state-of-the-art CS coding methods, such as SDPC and LSMM. On average, the proposed method achieves a PSNR gain of 1dB to 3dB across various sampling ratios. Qualitatively, CSCNet preserves significantly more structural detail and sharper edges. While previous methods often produce blurred or blocky artifacts at low bitrates, CSCNet maintains high visual fidelity.

Computational Efficiency

By utilizing a feed-forward neural network for reconstruction instead of iterative optimization, CSCNet achieves significantly faster inference speeds. This makes the framework highly suitable for practical, real-time compressive imaging systems where low latency is critical.

Conclusion and Future Work

CSCNet represents a significant step forward in image compressed sensing coding. By making the sampling process learnable and locally structured, the framework produces measurements that are highly compatible with existing image codecs. Combined with a multi-scale Laplacian pyramid reconstruction network, CSCNet achieves superior reconstruction quality and speed.

Despite its success, some limitations remain. Standard codecs use non-differentiable quantization steps, requiring approximations during training that may limit optimal efficiency. Additionally, while CSCNet excels at low to medium bitrates, further work is needed to surpass advanced traditional codecs at very high bitrates. Future research will focus on refining the end-to-end training process to better handle quantization and improving performance for high-fidelity applications.

Summary updated

Abstract

Existing image compressed sensing (CS) coding frameworks usually solve an inverse problem based on measurement coding and optimization-based image reconstruction, which still exist the following two challenges: (1) the widely used random sampling matrix, such as the Gaussian Random Matrix (GRM), usually leads to low measurement coding efficiency, and (2) the optimization-based reconstruction methods generally maintain a much higher computational complexity. In this article, we propose a new convolutional neural network based image CS coding framework using local structural sampling (dubbed CSCNet) that includes three functional modules: local structural sampling, measurement coding, and Laplacian pyramid reconstruction. In the proposed framework, instead of GRM, a new local structural sampling matrix is first developed, which is able to enhance the correlation between the measurements through a local perceptual sampling strategy. Besides, the designed local structural sampling matrix can be jointly optimized with the other functional modules during the training process. After sampling, the measurements with high correlations are produced, which are then coded into final bitstreams by the third-party image codec. Last, a Laplacian pyramid reconstruction network is proposed to efficiently recover the target image from the measurement domain to the image domain. Extensive experimental results demonstrate that the proposed scheme outperforms the existing state-of-the-art CS coding methods while maintaining fast computational speed.
BibTeX
@article{cui2024deep,
  author = {Wenxue Cui and Xingtao Wang and Xiaopeng Fan and Shaohui Liu and Xinwei Gao and Debin Zhao},
  title = {{Deep Network for Image Compressed Sensing Coding Using Local Structural Sampling}},
  journal = {ACM Trans. Multim. Comput. Commun. Appl.},
  year = {2024},
  url = {https://openreview.net/forum?id=SLayjHenm6},
}
Figure