Hardware-Aware Neural Architecture Search

1,639 concepts961 papers46 proposals

Deep learning models are traditionally designed in a "hardware vacuum," where researchers optimize for mathematical metrics like Top-1 accuracy or FLOPs (Floating Point Operations). However, as Capra et al. note, these metrics are poor proxies for real-world performance. A model with fewer FLOPs might actually run slower on a GPU if its operations do not align with the hardware's memory hierarchy or parallel processing capabilities.

Hardware-Aware Neural Architecture Search (HW-NAS) bridges this gap by incorporating physical constraints—latency, energy consumption, and memory footprint—directly into the automated design process.

The Proxy Problem: Predicting Performance

The primary challenge in HW-NAS is the "evaluation bottleneck." A search algorithm might explore thousands of candidate architectures; measuring each one on physical hardware is computationally prohibitive. To solve this, researchers use two main strategies to estimate performance without real-time benchmarking:

  1. Latency Lookup Tables: Chu et al. and Zhang et al. demonstrate that by pre-measuring the latency of individual building blocks (e.g., a 3x3 depthwise convolution with a specific stride) on a target device, the total model latency can be accurately predicted by summing the costs of its constituent parts.
  2. Predictive Modeling: Instead of simple summation, Wang and Magalhães et al. propose training secondary machine learning models to predict a candidate's latency or power consumption based on its architectural features. This allows the search engine to receive immediate feedback on how a change in width or depth will impact deployment.

Constrained Optimization and Pareto Frontiers

Once performance can be predicted, the search becomes a multi-objective optimization problem. The goal is to find the Pareto frontier: the set of models where no model can be made more accurate without also becoming slower or more power-hungry.

Dong et al. show that platform-aware search strategies yield models that are significantly faster than those produced by accuracy-only search. However, Burrello et al. argue that simply minimizing latency is insufficient for edge devices like microcontrollers. They propose treating hardware limits (like a 320KB SRAM limit) as strict boundaries rather than minimization objectives. This ensures the search utilizes the full available hardware budget to maximize accuracy, rather than unnecessarily sacrificing performance for efficiency once the model already fits the device.

Hardware-Software Co-Design

The most advanced iteration of this field moves beyond optimizing the model for a fixed chip. Instead, it treats the hardware itself as a flexible parameter.

Zhang et al. and Qi et al. propose simultaneous co-design, where a reinforcement learning agent explores a joint search space of neural network layers and FPGA accelerator parameters (such as parallelism factors and tiling sizes). This alignment is critical; as Hatef et al. demonstrate, simply adjusting Transformer dimensions to align with GPU tiling and scheduling mechanisms can increase throughput by 39% without any loss in accuracy.

Amortizing Search Cost with Super-networks

Searching for a new architecture for every single hardware target (e.g., a different model for a mobile GPU vs. a low-power CPU) is expensive. To address this, Wang introduced the SuperTransformer (and more generally, the Super-network), a large model that contains many smaller "sub-networks" within it.

By training this weight-sharing super-network once, researchers can then use a hardware-aware search to "extract" the optimal sub-network for any specific device. Sahni et al. further refined this with CompOFA, using compound scaling heuristics to prune the search space, achieving a 216x speedup in model deployment across diverse platforms. This shift allows a single training process to support an entire ecosystem of hardware, from high-end servers to battery-less, energy-harvesting sensors Sampson.

History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_hardware_aware_neural_architecture_searc_b086203d,
  title = {hardware-aware neural architecture search},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/hardware-aware-neural-architecture-search/rdr_b086203d610940af995c6b1e0eb22275},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-22},
}

This exact version (version 1, frozen text):

@misc{lacuna_hardware_aware_neural_architecture_searc_b086203d_v1,
  title = {hardware-aware neural architecture search},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/hardware-aware-neural-architecture-search/rdr_b086203d610940af995c6b1e0eb22275/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-22},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake