Hardware Aware Neural Architecture Search

This direction is merged into another direction: hardware-aware neural architecture search. Its last version stays readable and citable here.

Version 1 of 1573 papers757 conceptsfrozen 2026-10-04

In the traditional deep learning workflow, researchers design architectures to minimize a single metric: validation error. Efficiency is often treated as an afterthought, measured by indirect proxies like the number of parameters or Floating Point Operations (FLOPs). However, a model with fewer FLOPs is not necessarily faster on a smartphone, and a model with fewer parameters might actually consume more energy on an FPGA.

Hardware-Aware Neural Architecture Search (HW-NAS) shifts the paradigm by treating physical constraints—latency, RAM, and power consumption—as primary optimization objectives alongside accuracy. By incorporating the target hardware directly into the design loop, HW-NAS discovers architectures that are "Pareto-optimal," providing the best possible performance for a specific hardware budget.

The FLOPs Fallacy

The fundamental motivation for HW-NAS is the realization that theoretical efficiency metrics do not correlate perfectly with real-world performance. As Ningning Ma and colleagues demonstrated in ShuffleNet V2, two models with identical FLOP counts can have vastly different runtimes due to memory access costs and degree of parallelism.

For instance, depthwise convolutions are theoretically efficient but can be slow on hardware that isn't optimized for fragmented memory access. To solve this, researchers like Alberto Ancilotto in XiNet argue for moving beyond "low-MACs" (Multiply-Accumulate operations) obsession toward metrics like arithmetic intensity and actual RAM usage, which more accurately reflect the bottlenecks of microcontroller-class hardware.

Hardware-in-the-Loop Optimization

The most direct way to ensure a model is efficient is to measure it on the device itself. This "hardware-in-the-loop" approach was popularized by Grace Chu and the team behind MobileNetV3. They used reinforcement learning where the "reward" for the search agent was a combination of accuracy and the actual measured latency on a mobile phone.

In the TinyML domain, Mani B. Srivastava describes a closed-loop development workflow where real-world power and memory metrics inform the search. This ensures that the resulting network doesn't just "fit" on a microcontroller but is optimized for its specific memory bandwidth and instruction set.

Because accuracy and efficiency are often in conflict, HW-NAS typically seeks a Pareto frontier: a set of models where no model can be made faster without losing accuracy. Jin-Dong Dong and others introduced PPP-Net, which uses a progressive search strategy to identify these trade-offs.

This multi-objective approach allows developers to pick the right model for the job. For example, in TimePPG, Matteo Risso and Alessio Burrello generated a family of models for heart-rate monitoring, allowing a developer to choose a 0.21 mJ model for a battery-constrained wearable or a slightly more accurate version for a plugged-in device.

Efficiency at Scale: From Microcontrollers to LLMs

While much of the early work focused on computer vision for mobile devices, the principles of HW-NAS are now being applied to Large Language Models (LLMs) and specialized networking hardware.

  • Inference-Efficient LLMs: Shivaram Venkataraman and his team in Scaling Laws Meet Model Architecture propose "conditional scaling laws." By optimizing architectural hyperparameters like MLP-to-attention ratios, they found LLMs that are 42% faster during inference than standard baselines like LLaMA-3.2 without losing accuracy.
  • Networking Hardware: In Homunculus, Tushar Swamy demonstrates how Bayesian optimization can act as a "feasibility filter," discarding model configurations that exceed the strict memory limits of programmable switches before they are even trained.

Reducing the Search Cost

The main drawback of HW-NAS is the computational cost of searching through thousands of candidates. To address this, researchers have developed several "shortcuts":

  1. Supernets: Instead of training every candidate, researchers train a single "supernet" that contains all possible paths. As seen in SCAN-Edge and RaFL, specific sub-architectures can then be extracted and evaluated instantly.
  2. Training-Free Proxies: Shital Shah in LiteTransformerSearch uses evolutionary search on "zero-cost" proxies, which estimate a model's potential at initialization. This bypasses the need for expensive training cycles entirely.
  3. LLM-Guided Search: A recent trend, exemplified by FL-NAS by Ruiyang Qin and colleagues, uses LLMs like GPT-4 to reason about hardware constraints. By including peak memory and latency targets in the prompt, the LLM can suggest compact operations that naturally fit the device's limits.

By moving the hardware constraint from a post-hoc check to a core part of the optimization objective, HW-NAS ensures that the next generation of AI is not just powerful in the lab, but performant in the pocket.

Papers in this version

Showing 60 of 573 papers.

History & citationVersion 1 · 2026-10-04

Merged into another direction · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_hardware_aware_neural_architecture_searc_7bc2737e,
  title = {Hardware Aware Neural Architecture Search},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/hardware-aware-neural-architecture-search/rdr_7bc2737ea44e4b8fae80e86b05183364},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-22},
}

This exact version (version 1, frozen text):

@misc{lacuna_hardware_aware_neural_architecture_searc_7bc2737e_v1,
  title = {Hardware Aware Neural Architecture Search},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/hardware-aware-neural-architecture-search/rdr_7bc2737ea44e4b8fae80e86b05183364/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-22},
}