Low-Cost Image Compressive Sensing with Multiple Measurement Rates for Object Detection

Longlong Liao, Kenli Li, Canqun Yang, Jie Liu2019 · DOI

TL;DR

This paper introduces MRCS, a low-cost image compressive sensing (CS) approach for object detection in resource-constrained surveillance systems. By using multiple measurement rates (MRs) to sample key object regions at higher rates and other areas at lower rates, and employing half-precision floats for CS measurements, MRCS significantly reduces transmission bandwidth and storage space while maintaining reconstruction quality and object detection accuracy.

What Problem Does This Paper Address?

High-resolution cameras and drones are increasingly used for large-scale surveillance, generating massive amounts of image and video data. Transmitting and storing this data is a significant challenge, especially for resource-limited devices like surveillance cameras and aerial drones. Traditional compressive sensing (CS) methods often struggle with reconstruction quality when measurement rates are reduced, and they typically use a single measurement rate for the entire image, leading to inefficient sampling. This paper addresses the need for a low-cost, efficient CS approach that can reduce data overheads while preserving essential information for object detection in real-world surveillance scenarios.

Methods

The proposed MRCS approach has three main stages:

  1. Object Detection: To accelerate object detection on resource-constrained devices, the authors introduce MYOLO3, a smaller, efficient object detector. MYOLO3 is built upon bottleneck residual blocks and depthwise separable convolutions, aiming for a good trade-off between accuracy and computational cost.
  2. CS Sampling with Multiple Measurement Rates (MRs): This is the core innovation. Instead of a single MR, MRCS uses multiple MRs. Key regions identified by MYOLO3 are sampled with higher MRs to preserve detailed features, while less critical regions are sampled with lower MRs. This strategy reduces the overall size of the CS measurements.
  3. Half-Precision CS Measurements: To further reduce data size, the CS measurements are represented using 16-bit half-precision floats (FP16) instead of standard 32-bit single-precision floats. This is justified by research showing minimal degradation in DNN model accuracy when using FP16.
  4. CS Reconstruction: On high-performance servers, multiple CS reconstruction networks, each corresponding to a specific MR used during sampling, are employed to recover the natural images.

Datasets

  • Training MYOLO3: PASCAL VOC 2007 and 2012 trainval dataset (VOC2007+2012), containing 20 classes and 16,551 training images.
  • Testing MYOLO3: PASCAL VOC Test 2007 dataset, containing 4952 test images.
  • Training CS Reconstruction Model: A dataset of 91 images from a previous work [37].
  • Testing MRCS (CS Sampling & Reconstruction): A newly created test dataset named VOC-pbc, derived from 2171 images from PASCAL VOC Test 2007 that contain objects like persons, bicycles, and cars. This dataset is crucial for evaluating object detection performance on reconstructed images.
  • Evaluating AP on Reconstructed Images: A dataset named VOC-revise, created by replacing images in the original PASCAL VOC Test 2007 dataset with reconstructed images from the VOC-pbc dataset.

Key Insights and Ideas

  • Targeted Sampling is Key: The most significant insight is that not all parts of an image are equally important for object detection. By intelligently sampling key regions at higher rates and less critical areas at lower rates, substantial data reduction can be achieved without sacrificing detection performance.
  • Half-Precision for Efficiency: Leveraging half-precision floats for CS measurements is a straightforward yet effective way to halve the data size for transmission and storage, with minimal impact on reconstruction quality and downstream tasks like object detection.
  • MYOLO3 as a Lightweight Detector: The development of MYOLO3 demonstrates that efficient object detection models can be designed for resource-constrained environments, making them suitable for integration into CS pipelines on edge devices.
  • Multiple Reconstruction Networks: The need for multiple reconstruction networks, each tailored to a specific MR, highlights the complexity of reconstructing images sampled with varying rates.

Key Results

  • MYOLO3 Performance: Achieved the highest mAP of 74.0% on the PASCAL VOC 2007 test dataset among compared one-stage detectors, with a model size of only 4.80 million parameters.
  • Data Reduction: MRCS significantly reduces CS measurement size. For example, when MRs drop from 0.25 to 0.01, the mean Compression Ratio (mCR) increases by 125.87%.
  • Bandwidth Savings: Required network bandwidth is substantially reduced. For instance, bandwidth is reduced by 54.75% when MR₁ drops from 0.25 to 0.01.
  • Reconstruction Quality: Using half-precision floats for CS measurements results in nearly identical mPSNR and AP values compared to single-precision floats. For example, with MRs of 0.25/0.10, half-precision yields mCRs of 2.87 and bandwidth of 4.69 Mb/s, while single-precision yields mCRs of 1.43 and bandwidth of 9.38 Mb/s.
  • Object Detection Accuracy: MRCS maintains good object detection accuracy (AP) on reconstructed images, especially when higher MRs are used for key regions. For instance, AP for persons, bicycles, and cars increased by 64.50%, 177.05%, and 89.76% respectively when MRh increased from 0.04 to 0.25 (with MRl=0.01).

Limitations and Future Work

  • Limitations:
    • While MYOLO3 is smaller and more efficient than some detectors, it has more layers than MobileNet+SSD and PeleeNet, leading to higher computation cost in some aspects.
  • Reconstructed images with MR₁ = 1.00 (no CS) yield slightly lower APs than original images, suggesting some information loss even in the "full" reconstruction case.
  • The paper focuses on image CS; extending to video CS is mentioned as future work.
  • Future Work:
    • The authors suggest that the proposed framework can be readily extended to the CS reconstruction of videos.

Summary updated

BibTeX
@misc{liao2019low,
  author = {Longlong Liao and Kenli Li and Canqun Yang and Jie Liu},
  title = {{Low-Cost Image Compressive Sensing with Multiple Measurement Rates for Object Detection}},
  year = {2019},
  doi = {10.3390/s19092079},
}
Figure