Dynamic Temperature Management of Near-Sensor Processing for Energy-Efficient High-Fidelity Imaging

Venkatesh Kodukula, Saad Katrawala, Britton Jones, Carole-Jean Wu, Robert LiKamWaSensors 2021 · OpenReview

TL;DR

This paper tackles the challenge of overheating in near-sensor processing (NSP) for vision systems, which promises significant energy savings but degrades image quality. They introduce "Stagioni," a novel thermal management system that dynamically adjusts processing to balance energy efficiency with the varying fidelity demands of imaging tasks, achieving up to 53% power savings.

What Problem Does This Paper Address?

Traditional vision systems move image data from the sensor to a separate processing unit, which is energy-intensive due to data transfer. Pushing processing closer to the sensor (near-sensor processing, NSP) dramatically reduces this data movement, leading to substantial energy savings. However, this close integration generates heat, raising the sensor's temperature. Elevated sensor temperatures introduce noise into captured images, degrading their quality and potentially impacting the accuracy of vision tasks. This creates a fundamental trade-off: more processing near the sensor means more energy efficiency but worse image fidelity. The problem is how to achieve both energy efficiency and high-fidelity imaging simultaneously, especially when imaging needs can change dynamically.

Methods

The authors propose and evaluate two novel thermal management strategies, implemented within a runtime controller called "Stagioni":

  • Stop-Capture-Go: This policy temporarily halts near-sensor processing (entering a "CAP mode") to allow the sensor to cool down, enabling on-demand high-fidelity captures. Processing then resumes. This is architecturally simple and focuses on gating the near-sensor VPU and DRAM.
  • Seasonal Migration: This strategy involves shifting processing between a near-sensor VPU (for efficiency) and a thermally isolated far-sensor VPU (on the SoC) for high-fidelity captures. This allows for continuous processing while managing temperature.

These policies are designed based on three core principles:

  1. Situational Temperature Regulation: Keeping sensor temperature within a threshold suitable for current imaging fidelity needs.
  2. On-Demand Fidelity: Rapidly dropping temperature when high-fidelity imaging is requested.
  3. Duty Cycle Governs System Efficiency: Optimizing the balance between processing and cooling phases to minimize overall power consumption.

The authors developed an end-to-end modeling framework to characterize the energy, thermal, and noise implications of NSP. They also built an emulation framework using FPGA platforms to evaluate their proposed policies.

Datasets

The paper does not explicitly mention training datasets. For evaluation, they use a "life-logger" use case scenario that involves continuous vision tasks (image classification and object detection) with occasional imaging. They simulate various ambient temperatures (20°C to 40°C) and lighting conditions (simulated by adjusting camera settings like exposure and ISO). They also use a dataset of noisy images generated by injecting Gaussian thermal noise to evaluate task accuracy degradation.

Key Insights and Ideas

  • Rapid Temperature Drop is Key: A crucial insight is that disabling near-sensor processing leads to a very rapid drop in sensor junction temperature (on the order of milliseconds). This "thermal slack" is a unique opportunity for dynamic thermal management, unlike traditional CPU thermal management which focuses on steady-state limits.
  • Imaging Fidelity is Dynamic: The required image fidelity is not constant. It depends on the application (e.g., human consumption vs. machine vision), environmental conditions (lighting, ambient temperature), and specific events. This dynamism is central to their approach.
  • NSP Power Directly Impacts Temperature: The paper quantifies the direct relationship between near-sensor processing power and sensor temperature, showing that even small reductions in power lead to significant temperature drops.
  • Thermal Noise is a Major Bottleneck: The research clearly demonstrates that increased sensor temperature directly leads to increased thermal noise, visibly degrading image quality and reducing task accuracy.
  • Stop-Capture-Go is Power-Efficient but Performance-Limited: While Stop-Capture-Go offers the lowest power consumption, it can significantly impact performance due to frame drops, making it suitable for non-performance-critical applications.

Key Results

  • Significant Power Savings: The proposed Stagioni policies (Stop-Capture-Go and Seasonal Migration) achieve system power savings ranging from 22% to 53% compared to traditional full-far processing.
  • Negligible Performance Impact (for Seasonal Migration): Seasonal Migration, with its minimal switching overhead (100 µs), shows negligible impact on vision application performance.
  • Adaptive Thermal Management: Stagioni effectively adapts thermal boundaries based on ambient temperature and lighting conditions, ensuring appropriate temperature regulation for varying fidelity needs.
  • Thermal Model Validation: Their RC-based thermal model closely matches real sensor measurements with an error margin of 0.1%.
  • Rapid Temperature Drop Quantified: Disabling near-sensor processing can reduce sensor temperature by up to 13°C within 20 ms.

Limitations and Future Work

  • Emulation-Based: The evaluation is primarily based on an emulation framework due to the lack of readily available programmable 3D stacked image sensors. While they validate their thermal models with real sensor measurements, the full system evaluation is simulated.
  • Stop-Capture-Go Performance Trade-off: The Stop-Capture-Go policy, while highly power-efficient, can lead to significant performance degradation due to frame drops, especially when high fidelity or continuous processing is required.
  • Future Work: The authors suggest that their work is an early step towards imaging-aware dynamic thermal management. Future work could explore more sophisticated runtime controllers, hardware implementations, and a wider range of applications and sensor technologies. They also highlight the need for further research into mitigating the switching overhead for seasonal migration in more complex scenarios.

Summary updated

Abstract

Vision processing on traditional architectures is inefficient due to energy-expensive off-chip data movement. Many researchers advocate pushing processing close to the sensor to substantially reduce data movement. However, continuous near-sensor processing raises sensor temperature, impairing imaging/vision fidelity. We characterize the thermal implications of using 3D stacked image sensors with near-sensor vision processing units. Our characterization reveals that near-sensor processing reduces system power but degrades image quality. For reasonable image fidelity, the sensor temperature needs to stay below a threshold, situationally determined by application needs. Fortunately, our characterization also identifies opportunities—unique to the needs of near-sensor processing—to regulate temperature based on dynamic visual task requirements and rapidly increase capture quality on demand. Based on our characterization, we propose and investigate two thermal management strategies—stop-capture-go and seasonal migration—for imaging-aware thermal management. For our evaluated tasks, our policies save up to 53% of system power with negligible performance impact and sustained image fidelity.
BibTeX
@inproceedings{kodukula2021dynamic,
  author = {Venkatesh Kodukula and Saad Katrawala and Britton Jones and Carole-Jean Wu and Robert LiKamWa},
  title = {{Dynamic Temperature Management of Near-Sensor Processing for Energy-Efficient High-Fidelity Imaging}},
  booktitle = {Sensors},
  year = {2021},
  url = {https://openreview.net/forum?id=AuhaR9rMje},
}
Figure