Device-system Co-design of Photonic Neuromorphic Processor using Reinforcement Learning

Yingheng Tang, Princess Tara Zamani, Ruiyang Chen, Jianzhu Ma, Minghao Qi, Cunxi Yu, Weilu GaoarXiv 2022 · OpenReview

Introduction

Photonic neuromorphic processors represent a frontier in high-performance computing, offering the potential for extreme speed and energy efficiency in machine learning (ML) tasks. However, a significant "design gap" persists between device engineering and system architecture. Traditionally, device engineers optimize for physical metrics like modulation depth or transmission, while system architects focus on end-to-end performance metrics like inference accuracy. This disconnect often results in hardware that, while theoretically superior, performs poorly in real-world systems due to noise, quantization errors, and non-linearities.

This research introduces a "top-down" co-design methodology that utilizes Reinforcement Learning (RL) to bridge this gap. By optimizing the physical structure of a photonic processor based directly on system-level performance, the authors demonstrate a hardware accelerator capable of high-accuracy image classification and materials discovery.

The O-GEMM Accelerator and SRA

The core of the proposed system is a 3D free-space optical general matrix multiplication (O-GEMM) accelerator. The hardware utilizes a Spatially Reconfigurable Array (SRA) composed of multilayer thin films. These films incorporate Germanium-Antimony-Tellurium (GST), a chalcogenide phase-change material (PCM) that enables non-volatile, "in-memory" optical computing. By switching between amorphous and crystalline states, the SRA allows for the modulation of light to perform complex matrix operations directly in the optical domain.

Reinforcement Learning for Hardware Optimization

To optimize the SRA hardware, the researchers implemented three distinct RL strategies to explore the design space of layer materials and thicknesses:

  1. Deep Q-Learning (DQNN): A neural network acts as an automated designer, iteratively suggesting structural changes to maximize a reward based on GEMM calculation accuracy.
  2. Bayesian Optimization (BO): A probabilistic approach used to efficiently navigate the complex design space.
  3. Cascaded Approach: By using BO to establish a high-quality starting point for the DQNN, the authors significantly accelerated convergence and achieved superior final designs compared to using either method in isolation.

The "reward" for these agents was derived from a highly parallelized hardware emulator. This emulator incorporated "physics-aware training," which accounts for hardware-specific constraints such as detector noise and finite-bit quantization (approximately 5-bit precision). This ensures that the resulting software models are robust enough to handle the inherent imperfections of analog hardware.

Benchmarking and Results

The co-designed processor was evaluated across several standard and domain-specific benchmarks, including MNIST, Fashion-MNIST (F-MNIST), and Kuzushiji-MNIST (K-MNIST). Additionally, the system was applied to a "Materials Discovery" task using the Computational 2D Materials Database (C2DB) to predict the magnetic properties of 2D materials.

The results demonstrate that the co-designed O-GEMM system performs remarkably close to ideal digital implementations:

  • MNIST Accuracy: The system achieved 96.48% accuracy, nearly matching the 97.17% baseline of a standard GPU.
  • Materials Discovery: The processor reached 86.10% accuracy, identical to the GPU-based baseline.
  • Robustness: Physics-aware training proved essential; without it, accuracy on MNIST dropped significantly (from 97% to 92%). Fine-tuning with the hardware emulator recovered nearly all performance loss.

Key Insights and Emergent Physics

One of the most compelling findings was that the RL agents, despite having no prior knowledge of optical physics, "discovered" physical principles through optimization. High-accuracy GEMM calculations were found to correlate strongly with high light transmission, significant modulation depth, and thin device structures.

Furthermore, the authors demonstrated a "closed-loop" design where the O-GEMM hardware was used to accelerate the RL process for designing subsequent generations of optical hardware. This methodology lowers the barrier for designing high-performance photonic devices, allowing the RL agent to manage the complex interplay of material properties and thin-film interference.

Conclusion and Future Outlook

While the current work focused on a 6-layer stack in a free-space configuration, future research will look toward more complex architectures and integration into silicon photonic integrated circuits (PICs) for commercial scalability. Implementing on-chip RL for real-time adaptation to environmental changes or hardware aging remains a promising next step for the field, potentially leading to fully autonomous, self-optimizing photonic computing systems.

Summary updated

Abstract

The incorporation of high-performance optoelectronic devices into photonic neuromorphic processors can substantially accelerate computationally intensive operations in machine learning (ML) algorithms. However, the conventional device design wisdom is disconnected with system optimization. We report a device-system co-design methodology to optimize a free-space optical general matrix multiplication (GEMM) hardware accelerator by engineering a spatially reconfigurable array made from chalcogenide phase change materials. With a highly-parallelized hardware emulator constructed based on experimental information, we demonstrate the design of unit device by optimizing GEMM calculation accuracy via reinforcement learning, including deep Q-learning neural network, Bayesian optimization, and their cascaded approach, which show a clear correlation between system performance metrics and physical device specifications. Furthermore, we employ physics-aware training approaches to deploy optimized hardware to the tasks of image classification, materials discovery, and a closed-loop design of optical ML accelerators. The demonstrated framework offers insights into the co-design of optoelectronic devices and systems with reduced human-supervision and domain-knowledge barriers.
BibTeX
@misc{tang2022device,
  author = {Yingheng Tang and Princess Tara Zamani and Ruiyang Chen and Jianzhu Ma and Minghao Qi and Cunxi Yu and Weilu Gao},
  title = {{Device-system Co-design of Photonic Neuromorphic Processor using Reinforcement Learning}},
  year = {2022},
  eprint = {2203.06061},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2203.06061},
}
Figure