DOCTOR: Dynamic On-Chip Temporal Variation Remediation Toward Self-Corrected Photonic Tensor Accelerators

Haotian Lu, Sanmitra Banerjee, Jiaqi Gu

Abstract

Photonic computing has emerged as a promising solution for accelerating computation-intensive artificial intelligence (AI) workloads, offering unparalleled speed and energy efficiency, especially in resource-limited, latency-sensitive edge computing environments. However, the deployment of analog photonic tensor accelerators encounters reliability challenges due to hardware noise and environmental variations. While off-chip noise-aware training and on-chip training have been proposed to enhance the variation tolerance of optical neural accelerators with moderate, static noise, we observe a notable performance degradation over time due to temporally drifting variations, which requires a real-time, in-situ calibration mechanism. To tackle this challenging reliability issues, for the first time, we propose a lightweight dynamic on-chip remediation framework, dubbed DOCTOR, providing adaptive, in-situ accuracy recovery against temporally drifting noise. The DOCTOR framework intelligently monitors the chip status using adaptive probing and performs fast in-situ training-free calibration to restore accuracy when necessary. Recognizing nonuniform spatial variation distributions across devices and tensor cores, we also propose a variation-aware architectural remapping strategy to avoid executing critical tasks on noisy devices. Extensive experiments show that our proposed framework can guarantee sustained performance under drifting variations with 34% higher accuracy and 2-3 orders-of-magnitude lower overhead compared to state-of-the-art on-chip training methods. Our code is open-sourced at https://github.com/ScopeX-ASU/DOCTOR.

The Challenge of Temporal Drift

Analog photonic computing uses light instead of electricity to perform the massive matrix multiplications required by AI. While this is incredibly fast and energy-efficient, it is also notoriously fragile. The core components—Micro-Ring Resonators (MRRs)—are extremely sensitive to their environment. Even a tiny temperature shift (as small as 0.05K) or electrical crosstalk can cause the "weights" stored on the chip to drift, leading to a total collapse in AI model accuracy.

Temporal drift and spatial noise distribution significantly degrade photonic accelerator accuracy over time.

Existing solutions either try to "toughen up" models during initial training (which fails when noise changes over time) or use on-chip retraining. However, retraining is often too slow, power-hungry, and requires labeled data. There is a critical gap: the need for a real-time, data-free, and lightweight way to fix these drifting errors while the chip is actually in use.

The DOCTOR Framework

The authors introduce the DOCTOR (Dynamic On-Chip Temporal Variation Remediation) framework, which provides adaptive, in-situ accuracy recovery. The framework consists of three main pillars:

  1. Salience-Aware Sparse Calibration: Instead of retraining the entire neural network, DOCTOR treats calibration as a local regression problem. It identifies which weight blocks are most "salient" (important to the final output) and prioritizes fixing those first. Crucially, this is data-free: it probes the hardware by sending identity matrices through the chip to see how the output is distorted, then calculates a quick correction.
  2. Variation-Aware Tile Remapping: Recognizing that noise isn't spread evenly across a chip (e.g., one corner might be hotter than another), DOCTOR solves a "Linear Assignment Problem." It dynamically moves the most important parts of the AI model to the "cleanest" (least noisy) physical cores on the chip.
  3. Adaptive Remediation Controller: To avoid wasting energy, a controller periodically monitors temperature sensors and error rates. It only triggers a "remediation" (a fix) when the drift exceeds a specific threshold, balancing the trade-off between accuracy and system throughput.
DOCTOR's dynamic remediation process restores accuracy during execution through salience-aware calibration and variation-aware remapping.

Experimental Results and Efficiency

The framework was evaluated using standard computer vision benchmarks (Fashion-MNIST, CIFAR-10, CIFAR-100) mapped onto simulated photonic hardware. The authors simulated various "drifting" scenarios, including linear temperature increases and "corner hotspots."

Key results include:

  • Accuracy Recovery: In scenarios where drifting noise crashed model accuracy from ~91% to ~57%, DOCTOR restored it to over 90% in just a few steps.
  • Superiority over SOTA: Compared to state-of-the-art on-chip training methods, DOCTOR provided 34% higher accuracy on average.
  • Massive Efficiency Gains: DOCTOR is 2 to 3 orders of magnitude (100x–1000x) faster and less resource-intensive than retraining-based methods.
  • Low Overhead: The framework only adds between 0.14% and 5% to the total cycle count of the inference process, making it viable for real-world edge deployment.

Conclusion

DOCTOR represents a significant step toward reliable, high-performance photonic AI accelerators. By decoupling calibration from training data and exploiting spatial noise intelligence, it achieves robust performance with minimal overhead. While currently optimized for MRR-based architectures, the principles of salience-aware calibration and variation-aware remapping offer a blueprint for future self-correcting analog hardware.

Code available at: https://github.com/ScopeX-ASU/DOCTOR

Summary updated

Abstract

Photonic computing has emerged as a promising solution for accelerating computation-intensive artificial intelligence (AI) workloads, offering unparalleled speed and energy efficiency, especially in resource-limited, latency-sensitive edge computing environments. However, the deployment of analog photonic tensor accelerators encounters reliability challenges due to hardware noise and environmental variations. While off-chip noise-aware training and on-chip training have been proposed to enhance the variation tolerance of optical neural accelerators with moderate, static noise, we observe a notable performance degradation over time due to temporally drifting variations, which requires a real-time, in-situ calibration mechanism. To tackle this challenging reliability issues, for the first time, we propose a lightweight dynamic on-chip remediation framework, dubbed DOCTOR, providing adaptive, in-situ accuracy recovery against temporally drifting noise. The DOCTOR framework intelligently monitors the chip status using adaptive probing and performs fast in-situ training-free calibration to restore accuracy when necessary. Recognizing nonuniform spatial variation distributions across devices and tensor cores, we also propose a variation-aware architectural remapping strategy to avoid executing critical tasks on noisy devices. Extensive experiments show that our proposed framework can guarantee sustained performance under drifting variations with 34% higher accuracy and 2-3 orders-of-magnitude lower overhead compared to state-of-the-art on-chip training methods. Our code is open-sourced at https://github.com/ScopeX-ASU/DOCTOR.
BibTeX
@article{lu2024doctor,
  author = {Haotian Lu and Sanmitra Banerjee and Jiaqi Gu},
  title = {{DOCTOR: Dynamic On-Chip Temporal Variation Remediation Toward Self-Corrected Photonic Tensor Accelerators}},
  journal = {Journal of Lightwave Technology},
  year = {2024},
  url = {https://openreview.net/forum?id=Tsk2OXjXOJ},
}
Figure