Multi-Cross Sampling and Frequency-Division Reconstruction for Image Compressed Sensing

Heping Song, Jingyao Gong, Hongying Meng, Yuping LaiAAAI 2024 · DOI

TL;DR

This paper introduces MCFD-Net, a novel deep learning framework for compressed sensing (CS) that significantly improves image reconstruction quality. By employing a new multi-cross sampling module (DMCS) to better capture high-level features during measurement acquisition and a frequency-division reconstruction module (FDRM) to separately process low and high frequencies, MCFD-Net achieves state-of-the-art performance with enhanced robustness to noise and lower sampling rates.

What Problem Does This Paper Address?

Compressed Sensing (CS) is a powerful technique that allows for signal acquisition at sub-Nyquist rates, enabling efficient data compression and reconstruction. However, traditional CS methods, and even many recent deep learning-based approaches (Deep Compressed Sensing or DCS), face limitations. A key challenge is that current DCS methods often use simplistic convolutional downsampling for acquiring measurements, which can lead to the loss of crucial high-level image features. This makes it difficult to achieve high-quality reconstructions, especially at very low sampling rates. Furthermore, these methods often treat all frequency components of the signal uniformly during reconstruction, neglecting the distinct roles of high and low frequencies in image detail and structure. This paper aims to address these shortcomings by developing a more sophisticated sampling and reconstruction strategy that preserves essential features and leverages frequency information for improved image recovery.

Methods

The proposed solution is a novel end-to-end deep learning network called MCFD-Net, which consists of two main novel modules:

  • Dynamic Multi-Cross Sampling (DMCS) Module: This module acts as the sensing network. It goes beyond simple convolution by employing:
  • Pyramid Cross Convolution: To capture features at multiple receptive field sizes, allowing for a richer understanding of the input signal.
  • Dual-Branch Sampling with Multi-Level Pooling: This helps retain feature information that might be lost in traditional pooling operations.
  • Attention Mechanism: Integrated between perception blocks, this mechanism adaptively enhances important feature channels, improving the learning process. The DMCS module is designed to intelligently acquire measurements, preserving high-level features for better downstream reconstruction.

  • Frequency-Division Reconstruction Module (FDRM): This module handles the deep reconstruction. Its key innovations include:

  • Discrete Wavelet Transform (DWT): Used to decompose the feature maps into distinct low-frequency (structural) and high-frequency (detail) components.
  • Frequency-Specific Processing: Multi-scale convolutions are applied to each frequency band separately.
  • Self-Similarity Attention Compensation: This is used to address local correlations lost during convolution, particularly important for high-frequency details.
  • Dynamic Attention Block: This block learns to fuse the processed low and high-frequency features, considering their respective contributions. The FDRM aims to reconstruct the image by intelligently combining information from different frequency domains, leading to more accurate and detailed results.

MCFD-Net integrates these modules into a cohesive framework, allowing for end-to-end training.

Datasets

The paper utilizes the following datasets:

  • COCO 2014: 40,000 sub-images (96x96) were used for training. Images were converted to the YCbCr color space, and only the Y channel was used for training and testing.
  • Set11: Used for evaluation.
  • BSDS100: Used for evaluation.
  • Urban100: Used for evaluation, particularly for assessing noise robustness.

No new benchmarks were introduced; the focus was on evaluating the proposed method against existing state-of-the-art techniques on established datasets.

Key Insights and Ideas

  • Feature Preservation is Crucial at Sampling: The paper strongly emphasizes that the quality of reconstruction is heavily dependent on how well features are captured during the sampling phase. Simple downsampling is insufficient; a more intelligent approach like DMCS is needed.
  • Frequency Domain is Not Uniform: Treating low and high frequencies identically during reconstruction is suboptimal. Separating and processing them differently, as done in FDRM, allows for better preservation of both structural integrity and fine details.
  • Attention Enhances Adaptability: The use of attention mechanisms in both sampling (DMCS) and reconstruction (FDRM) allows the network to dynamically focus on the most relevant features and frequency components, leading to improved performance.
  • Sampling Module Dominance: Ablation studies suggest that improvements in the sampling module (DMCS) can yield greater benefits than solely optimizing the reconstruction module, especially when computational complexity is a concern. This highlights the importance of a well-designed sensing strategy.

Key Results

  • State-of-the-Art Performance: MCFD-Net consistently outperforms existing methods on benchmark datasets (Set11, BSDS100, Urban100) across various sampling rates (from 50% down to 1.5625%).
  • Significant PSNR Gains: For example, on the Set11 dataset at 25% sampling rate, MCFD-Net achieves average PSNR gains of 1.72dB over MR-CCSNet+ and 1.07dB over CASNet. On BSDS100 at 25%, it shows improvements of 1.48dB over TransCS and 0.92dB over OCTUF.
  • Superior Visual Quality: Qualitative results (Figures 5 and 6) demonstrate that MCFD-Net produces reconstructions with fewer artifacts, less noise, and sharper details, especially at extremely low sampling rates where other methods fail.
  • Excellent Noise Robustness: Figure 7 and Table 2 show that MCFD-Net exhibits minimal performance degradation under varying levels of Gaussian noise, outperforming other methods in noise attenuation.
  • Efficient Reconstruction: The method achieves competitive inference times, reconstructing a 128x128 image in approximately 65ms on a GPU, comparable to other methods with similar parameter counts.

Limitations and Future Work

  • Limitations:
    • The paper acknowledges that while MCFD-Net offers a good trade-off, there is still a slight increase in computing parameters compared to some simpler methods, though this is deemed justifiable by the performance gains.
  • The current work focuses on still image CS.
  • Future Work:
    • The authors plan to extend MCFD-Net to video compressed sensing.
  • They also aim to apply the framework to other inverse problems.

Summary updated

BibTeX
@inproceedings{song2024multi,
  author = {Heping Song and Jingyao Gong and Hongying Meng and Yuping Lai},
  title = {{Multi-Cross Sampling and Frequency-Division Reconstruction for Image Compressed Sensing}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2024},
  doi = {10.1609/aaai.v38i5.28294},
}
Figure