Abstract
The recent breakthroughs and prohibitive complexities of Deep Neural Networks (DNNs) have excited extensive interest in domain-specific DNN accelerators, among which optical DNN accelerators are particularly promising thanks to their unprecedented potential of achieving superior performance-per-watt. However, the development of optical DNN accelerators is much slower than that of electrical DNN accelerators. One key challenge is that while many techniques have been developed to facilitate the development of electrical DNN accelerators, techniques that support or expedite optical DNN accelerator design remain much less explored, limiting both the achievable performance and the innovation development of optical DNN accelerators. To this end, we develop the first-of-its-kind framework dubbed O-HAS, which for the first time demonstrates automated Optical Hardware Accelerator Search for boosting both the acceleration efficiency and development speed of optical DNN accelerators. Specifically, our O-HAS consists of two integrated enablers: (1) an O-Cost Predictor, which can accurately yet efficiently predict an optical accelerator's energy and latency based on the DNN model parameters and the optical accelerator design; and (2) an O-Search Engine, which can automatically explore the large design space of optical DNN accelerators and identify the optimal accelerators (i.e., the micro-architectures and algorithm-to-accelerator mapping methods) in order to maximize the target acceleration efficiency. Extensive experiments and ablation studies consistently validate the effectiveness of both our O-Cost Predictor and O-Search Engine as well as the excellent efficiency of O-HAS generated optical accelerators.
Introduction
Deep Neural Networks (DNNs) are computationally expensive, leading to a surge in domain-specific accelerators. Optical accelerators (OAs) are considered a potential "holy grail" in this field, promising orders of magnitude improvements in speed and power efficiency by utilizing photons instead of electrons for matrix computations. Despite this potential, OA development lags behind electrical counterparts due to high design complexity, a lack of standardized evaluation tools, and high application specificity. Currently, researchers spend months manually designing these chips, often missing the most efficient configurations within a massive design space.
The O-HAS Framework
To bridge this gap, we introduce O-HAS, the first automated framework designed to search for optimal optical hardware accelerators. O-HAS consists of two primary components that work in tandem to optimize design efficiency and performance.
O-Cost Predictor
The O-Cost Predictor is an analytical model that serves as a high-speed virtual simulator. It takes a DNN model and a proposed hardware design as input to estimate energy, latency, and area. Crucially, it supports three distinct types of optical computing units (OCUs):
- Z-OCU (MZI-based): Utilizes Mach-Zehnder Interferometers for coherent light math.
- E-OCU (EOM-based): Employs Electro-Optic Modulators and time-wavelength interleaving.
- R-OCU (MR-based): Uses Micro-Resonators for high-density, multi-wavelength computations.
O-Search Engine
The O-Search Engine automates the exploration of the design space using a Gumbel-Softmax strategy. It treats hardware parameters—such as memory size, tile count, and data mapping—as searchable variables. This allows the system to mathematically converge on the optimal design for a specific DNN workload, transforming a process that previously took months into one that takes minutes or hours.
Design Space and Key Insights
A critical finding of this work is the extreme difficulty of manual design. In a search space of over 140,000 possible configurations for a model like LeNet-5, fewer than 0.72% were found to be high-performing. This "needle in a haystack" problem underscores the necessity of automated search tools like O-HAS.
Furthermore, the research highlights the importance of hardware-software co-design. O-HAS does not just optimize physical parameters; it also searches for the best "loop-tiling" and "loop-order" to minimize data movement. The study also reveals technology sensitivity: while Micro-resonator (MR) designs excel in energy efficiency for large-scale DNNs, MZI-based designs offer high compute density but face scaling challenges due to their physical footprint.
Experimental Results
O-HAS was evaluated across standard benchmarks including LeNet-5, AlexNet, ResNet-18, and VGG-16. Key results include:
- Accuracy: The O-Cost Predictor achieved an average error of only ~8.5% for energy and ~12.5% for latency compared to cycle-accurate simulators.
- Performance Gains: O-HAS generated designs (e.g., "OHAS-TEA") improved compute density by 1.86x and throughput-per-energy by 27.68x over state-of-the-art manual designs.
- Inference Efficiency: For actual model execution, O-HAS designs achieved up to a 32.48x improvement in FPS and a 119.70x improvement in FPS-per-Watt.
- Development Speed: Optimal designs were identified in as little as 100 seconds for simpler models, drastically reducing development time.
Conclusion and Future Work
O-HAS demonstrates that automated search can significantly boost both the performance and development speed of optical DNN accelerators. While the current framework focuses primarily on optimizing Optical Convolution Units (OCUs) within a generic template, future work will explore full optical implementations of pooling and activation layers, as well as physical fabrication and validation on silicon photonic chips.
Summary updated