Abstract
Vision processing on traditional architectures is inefficient due to energy-expensive off-chip data movement. Pushing processing close to the sensor substantially reduces data movement but raises sensor temperature, impairing imaging fidelity. We characterize the thermal implications of 3D stacked image sensors with near-sensor vision processing units (VPUs). Our characterization reveals that while near-sensor processing (NSP) reduces system power, it degrades image quality. We propose Stagioni, a thermal management runtime employing "stop-capture-go" and "seasonal migration" strategies. Evaluation shows Stagioni saves up to 53% of system power with negligible performance impact and sustained image fidelity.
Introduction: The Efficiency-Fidelity Trade-off
In traditional mobile vision systems, moving raw pixel data from the camera sensor to a separate System-on-Chip (SoC) creates a significant energy bottleneck. 3D-stacked architectures, which layer a Vision Processing Unit (VPU) and DRAM directly beneath the image sensor, offer a solution by minimizing off-chip data movement.
However, this proximity introduces a "heat trap." Continuous processing by the VPU generates heat that elevates the sensor's temperature. High temperatures increase dark current shot noise, which degrades the signal-to-noise ratio (SNR). This noise not only makes images aesthetically unpleasing but also reduces the accuracy of computer vision algorithms, potentially leading to misclassifications.
Methodology: The Stagioni Runtime
To address these thermal challenges, we developed Stagioni, an imaging-aware thermal management runtime. Stagioni is built upon three technical pillars:
1. Thermal-Electrical Modeling
We utilized an RC (Resistor-Capacitor) duality model to characterize heat flow within the 3D-stacked sensor. By modeling the thermal resistance and capacitance of each layer (Sensor, DRAM, VPU, Substrate), we can predict temperature fluctuations based on power consumption. This model was validated against real-world hardware, specifically the OnSemi AR0330 sensor.
2. Stop-Capture-Go
This strategy is designed for systems that can tolerate occasional frame drops. It periodically gates the clock of the near-sensor VPU and DRAM, creating "temporal slack." This pause allows the sensor to cool down before resuming processing, ensuring that high-fidelity captures occur at lower temperatures.
3. Seasonal Migration
For applications requiring continuous, uninterrupted vision (e.g., life-logging), Stagioni employs seasonal migration. When the sensor temperature exceeds a dynamic threshold, the processing task is migrated from the efficient "near-sensor" VPU to a "far-sensor" VPU located on the SoC. Once the sensor has cooled sufficiently, the task migrates back to the more efficient unit. This approach maintains continuous operation while protecting image quality.
Key Insight: The "Immediate Drop" Phenomenon
A critical discovery in our characterization is the difference between package temperature and junction temperature. While the overall chip package takes seconds to cool, the junction temperature—the actual silicon interface—drops almost instantaneously when power is cut.
As shown in the figure above, the junction temperature can drop by as much as 13°C within just 20ms. Stagioni leverages this phenomenon to provide "on-demand" high-fidelity capture. By briefly pausing processing right before a high-quality photo is taken, the system can significantly reduce thermal noise with minimal impact on the overall processing pipeline.
Results and Evaluation
We evaluated Stagioni using a life-logger case study involving various Deep Neural Network (DNN) models, including AlexNet, MobileNetSSD, GoogLeNet, and ResNet50.
- Energy Savings: Stagioni achieved system power savings between 22% and 53% compared to traditional off-sensor processing architectures.
- Image Fidelity: The system successfully maintained target image quality (e.g., 35dB SNR) even under heavy computational loads.
- Low Overhead: The seasonal migration strategy proved highly efficient, with a switching latency of only 100µs. This is negligible compared to standard 30fps frame capture times (33ms), resulting in zero dropped frames during the migration process.
Conclusion
Stagioni demonstrates that the thermal challenges of near-sensor processing are manageable through intelligent, imaging-aware runtime strategies. By exploiting the rapid thermal dynamics of the silicon junction and utilizing situational awareness (such as ambient light levels), Stagioni unlocks the energy-efficiency potential of 3D-stacked sensors without sacrificing the high-fidelity imaging required for modern mobile and wearable vision applications.
Summary updated