O-HAS: Optical Hardware Accelerator Search for Boosting Both Acceleration Performance and Development Speed

Mengquan Li, Zhongzhi Yu, Yongan Zhang, Yonggan Fu, Yingyan Lin

TL;DR

O-HAS is the first automated framework designed to search for optimal optical hardware accelerators (OAs) for Deep Neural Networks. By combining a fast analytical cost predictor with a differentiable search engine, it identifies hardware architectures that outperform manual designs by up to 119x in efficiency while slashing development time from months to minutes.

What Problem Does This Paper Address?

Deep Neural Networks (DNNs) are computationally expensive, leading to a surge in domain-specific accelerators. Optical accelerators (OAs) are the "holy grail" of this field, promising 1000x faster speeds and 1000x lower power than electronics by using light (photons) instead of electricity (electrons) for matrix math.

However, OA development is lagging behind its electrical counterparts for three main reasons:

  1. Design Complexity: Creating an OA requires cross-disciplinary knowledge (optics + computer architecture), and the design space is massive.
  2. Lack of Tools: There are no "fair" ways to compare different optical technologies (like MZIs vs. Micro-resonators) before actually building them.
  3. Application Specificity: A design that works perfectly for a small model like LeNet-5 might be terrible for a massive model like VGG-16.

Currently, researchers spend months manually designing these chips, often missing the most efficient configurations.

Methods

The authors introduce O-HAS, a framework consisting of two main components:

  • O-Cost Predictor: This is an analytical model that acts as a "virtual simulator." It takes a DNN model and a proposed hardware design as input and quickly estimates the energy, latency, and area. It is unique because it can model three distinct types of optical computing units:
  • Z-OCU (MZI-based): Uses interferometers for coherent light math.
  • E-OCU (EOM-based): Uses modulators and time-wavelength interleaving.
  • R-OCU (MR-based): Uses tiny ring resonators for high-density, multi-wavelength math.
  • O-Search Engine: Instead of guessing designs, this engine uses a Gumbel-Softmax strategy—a technique borrowed from Neural Architecture Search (NAS). It treats hardware parameters (like memory size, number of tiles, and data mapping) as "searchable" variables, allowing the system to mathematically converge on the best design for a specific DNN.

Datasets

The authors evaluated O-HAS across a variety of standard computer vision benchmarks to prove its versatility:

  • LeNet-5 (trained on MNIST) for simple digit recognition.
  • AlexNet, ZFNet, ResNet-18, GoogLeNet, and VGG-16 (trained on ImageNet) for complex object recognition.

These models represent a range of "shapes and sizes," testing the framework's ability to adapt hardware to different computational workloads.

Key Insights and Ideas

  • The "Needle in a Haystack" Problem: The authors found that in a search space of over 140,000 possible designs, fewer than 0.72% were actually high-performing (Fig 2). This explains why manual design is so difficult—humans are unlikely to stumble upon the optimal configuration by chance.
  • Hardware-Software Co-Design: The search isn't just about the physical chip; it’s about the mapping. O-HAS searches for the best "loop-tiling" and "loop-order," essentially deciding how the data flows through the light-based circuits to minimize data movement.
  • Technology Sensitivity: The results show that different optical technologies have different "sweet spots." For example, Micro-resonator (MR) based designs excel in energy efficiency for large-scale DNNs, whereas MZI-based designs offer high compute density but struggle to scale to larger networks due to their physical footprint.

Key Results

  • High Accuracy: The O-Cost Predictor achieved an average error of only ~8.5% for energy and ~12.5% for latency compared to slow, cycle-accurate simulators.
  • Massive Speedups: The O-HAS generated designs (specifically the "OHAS-TEA" variant) improved compute density by 1.86x and throughput-per-energy by 27.68x over state-of-the-art manual designs.
  • Inference Performance: When running actual models, O-HAS designs achieved up to a 32.48x improvement in Frames Per Second (FPS) and a staggering 119.70x improvement in FPS-per-Watt.
  • Development Speed: The search engine can find these optimal designs in seconds to hours (e.g., ~100 seconds for LeNet-5), a task that previously took human experts months.

Limitations and Future Work

  • Focus on Convolutions: The paper primarily focuses on optimizing the Optical Convolution Units (OCUs). While it accounts for peripheral electronics (like ADCs/DACs), it leaves the full optical implementation of pooling and activation layers to future work.
  • Generic Taxonomy: The search is limited to the "generic optical accelerator" template defined by the authors. While broad, it may not capture radical, non-standard optical computing architectures.
  • Physical Realization: As with all architectural search papers, the results are based on modeling and simulation. The next step is validating these "optimal" designs through physical fabrication on silicon photonic chips.

Summary updated

Abstract

The recent breakthroughs and prohibitive complexities of Deep Neural Networks (DNNs) have excited extensive interest in domain specific DNN accelerators, among which optical DNN accelerators are particularly promising thanks to their unprecedented potential of achieving superior performance-per-watt. However, the development of optical DNN accelerators is much slower than that of electrical DNN accelerators. One key challenge is that while many techniques have been developed to facilitate the development of electrical DNN accelerators, techniques that support or expedite optical DNN accelerator design remain much less explored, limiting both the achievable performance and the innovation development of optical DNN accelerators. To this end, we develop the first-of-its-kind framework dubbed O-HAS, which for the first time demonstrates automated Optical Hardware Accelerator Search for boosting both the acceleration efficiency and development speed of optical DNN accelerators. Specifically, our O-HAS consists of two integrated enablers: (1) an O-Cost Predictor, which can accurately yet efficiently predict an optical accelerator's energy and latency based on the DNN model parameters and the optical accelerator design; and (2) an O-Search Engine, which can automatically explore the large design space of optical DNN accelerators and identify the optimal accelerators (i.e., the micro-architectures and algorithm-to-accelerator mapping methods) in order to maximize the target acceleration efficiency. Extensive experiments and ablation studies consistently validate the effectiveness of both our O-Cost Predictor and O-Search Engine as well as the excellent efficiency of O-HAS generated optical accelerators.
BibTeX
@inproceedings{li2021o,
  author = {Mengquan Li and Zhongzhi Yu and Yongan Zhang and Yonggan Fu and Yingyan Lin},
  title = {{O-HAS: Optical Hardware Accelerator Search for Boosting Both Acceleration Performance and Development Speed}},
  booktitle = {ICCAD},
  year = {2021},
  url = {https://openreview.net/forum?id=2wtCoTAoR0},
}

Loading related content...

Figure