Stagioni: Temperature management to enable near-sensor processing for energy-efficient high-fidelity imaging

Venkatesh Kodukula, Saad Katrawala, Britton Jones, Carole-Jean Wu, Robert LiKamWaarXiv 2019 · DOI · arXiv

Abstract

Vision processing on traditional architectures is inefficient due to energy-expensive off-chip data movement. Pushing processing close to the sensor substantially reduces data movement but raises sensor temperature, impairing imaging fidelity. We characterize the thermal implications of 3D stacked image sensors with near-sensor vision processing units (VPUs). Our characterization reveals that while near-sensor processing (NSP) reduces system power, it degrades image quality. We propose Stagioni, a thermal management runtime employing "stop-capture-go" and "seasonal migration" strategies. Evaluation shows Stagioni saves up to 53% of system power with negligible performance impact and sustained image fidelity.

Introduction: The Efficiency-Fidelity Trade-off

In traditional mobile vision systems, moving raw pixel data from the camera sensor to a separate System-on-Chip (SoC) creates a significant energy bottleneck. 3D-stacked architectures, which layer a Vision Processing Unit (VPU) and DRAM directly beneath the image sensor, offer a solution by minimizing off-chip data movement.

However, this proximity introduces a "heat trap." Continuous processing by the VPU generates heat that elevates the sensor's temperature. High temperatures increase dark current shot noise, which degrades the signal-to-noise ratio (SNR). This noise not only makes images aesthetically unpleasing but also reduces the accuracy of computer vision algorithms, potentially leading to misclassifications.

Comparison of traditional vision pipelines versus 3D stacked sensors, highlighting the fidelity degradation caused by heat in near-sensor processing.

Methodology: The Stagioni Runtime

To address these thermal challenges, we developed Stagioni, an imaging-aware thermal management runtime. Stagioni is built upon three technical pillars:

1. Thermal-Electrical Modeling

We utilized an RC (Resistor-Capacitor) duality model to characterize heat flow within the 3D-stacked sensor. By modeling the thermal resistance and capacitance of each layer (Sensor, DRAM, VPU, Substrate), we can predict temperature fluctuations based on power consumption. This model was validated against real-world hardware, specifically the OnSemi AR0330 sensor.

2. Stop-Capture-Go

This strategy is designed for systems that can tolerate occasional frame drops. It periodically gates the clock of the near-sensor VPU and DRAM, creating "temporal slack." This pause allows the sensor to cool down before resuming processing, ensuring that high-fidelity captures occur at lower temperatures.

3. Seasonal Migration

For applications requiring continuous, uninterrupted vision (e.g., life-logging), Stagioni employs seasonal migration. When the sensor temperature exceeds a dynamic threshold, the processing task is migrated from the efficient "near-sensor" VPU to a "far-sensor" VPU located on the SoC. Once the sensor has cooled sufficiently, the task migrates back to the more efficient unit. This approach maintains continuous operation while protecting image quality.

Key Insight: The "Immediate Drop" Phenomenon

A critical discovery in our characterization is the difference between package temperature and junction temperature. While the overall chip package takes seconds to cool, the junction temperature—the actual silicon interface—drops almost instantaneously when power is cut.

Junction temperature trace showing the "immediate jump" phenomenon where temperature drops by 13°C within 20ms of power gating.

As shown in the figure above, the junction temperature can drop by as much as 13°C within just 20ms. Stagioni leverages this phenomenon to provide "on-demand" high-fidelity capture. By briefly pausing processing right before a high-quality photo is taken, the system can significantly reduce thermal noise with minimal impact on the overall processing pipeline.

Results and Evaluation

We evaluated Stagioni using a life-logger case study involving various Deep Neural Network (DNN) models, including AlexNet, MobileNetSSD, GoogLeNet, and ResNet50.

  • Energy Savings: Stagioni achieved system power savings between 22% and 53% compared to traditional off-sensor processing architectures.
  • Image Fidelity: The system successfully maintained target image quality (e.g., 35dB SNR) even under heavy computational loads.
  • Low Overhead: The seasonal migration strategy proved highly efficient, with a switching latency of only 100µs. This is negligible compared to standard 30fps frame capture times (33ms), resulting in zero dropped frames during the migration process.

Conclusion

Stagioni demonstrates that the thermal challenges of near-sensor processing are manageable through intelligent, imaging-aware runtime strategies. By exploiting the rapid thermal dynamics of the silicon junction and utilizing situational awareness (such as ambient light levels), Stagioni unlocks the energy-efficiency potential of 3D-stacked sensors without sacrificing the high-fidelity imaging required for modern mobile and wearable vision applications.

Summary updated

Abstract

Vision processing on traditional architectures is inefficient due to energy-expensive off-chip data movement. Many researchers advocate pushing processing close to the sensor to substantially reduce data movement. However, continuous near-sensor processing raises the sensor temperature, impairing the fidelity of imaging/vision tasks. We characterize the thermal implications of using 3D stacked image sensors with near-sensor vision processing units. Our characterization reveals that near-sensor processing reduces system power but degrades image quality. For reasonable image fidelity, the sensor temperature needs to stay below a threshold, situationally determined by application needs. Fortunately, our characterization also identifies opportunities -- unique to the needs of near-sensor processing -- to regulate temperature based on dynamic visual task requirements and rapidly increase capture quality on demand. Based on our characterization, we propose and investigate two thermal management strategies -- stop-capture-go and seasonal migration -- for imaging-aware thermal management. We present parameters that govern the policy decisions and explore the trade-offs between system power and policy overhead. Our evaluation shows that our novel dynamic thermal management strategies can unlock the energy-efficiency potential of near-sensor processing. For our evaluated tasks, our strategies save up to 53% of system power with negligible performance impact and sustained image fidelity.
BibTeX
@misc{kodukula2019stagioni,
  author = {Venkatesh Kodukula and Saad Katrawala and Britton Jones and Carole-Jean Wu and Robert LiKamWa},
  title = {{Stagioni: Temperature management to enable near-sensor processing for energy-efficient high-fidelity imaging}},
  year = {2019},
  eprint = {2001.01580},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2001.01580},
}
Figure