ESM: A Framework for Building Effective Surrogate Models for Hardware-Aware Neural Architecture Search

Azaz-Ur-Rehman Nasir, Samroz Ahmad Shoaib, Muhammad Abdullah Hanif, Muhammad Shafique

Abstract

Hardware-Aware Neural Architecture Search (HW-NAS) is essential for deploying deep learning models on resource-constrained devices, yet the cost of evaluating candidates on physical hardware remains a significant bottleneck. This paper introduces the Effective Surrogate Models (ESM) framework, which utilizes a novel Feature Combination Count (FCC) encoding and a balanced sampling strategy to predict model latency with high precision. ESM achieves up to 99% prediction accuracy while requiring 8x less training data than traditional methods, significantly accelerating the NAS process across diverse hardware platforms.

1. Introduction

Designing Deep Neural Networks (DNNs) for resource-constrained environments—such as mobile GPUs, edge CPUs, or IoT devices—requires a delicate balance between predictive accuracy and hardware efficiency (latency and energy consumption). While HW-NAS automates the discovery of optimal architectures, the evaluation phase is prohibitively slow; measuring thousands of candidate models on actual hardware can take weeks of compute time.

To mitigate this, researchers employ surrogate models: machine learning predictors trained to estimate a model's latency instantly. However, existing surrogate models often struggle with three major issues: inaccurate encodings (e.g., hardware-agnostic FLOPs), data inefficiency (the high cost of profiling), and high sparsity in architecture representation. The ESM framework is designed to overcome these hurdles through smarter sampling and a more descriptive encoding scheme.

2. The ESM Framework

The ESM framework optimizes the surrogate modeling process through three integrated pillars:

2.1 Balanced Sampling

Traditional random sampling of architectures often falls victim to the "central limit theorem" bias. In a large search space, random selection tends to pick models of average depth and complexity, creating a Gaussian distribution. Consequently, predictors become experts at predicting "average" models but fail to learn the behavior of very small or very large "corner-case" models that often define the Pareto front. ESM solves this by dividing the search space into "bins" based on model depth and ensuring an equal number of models are sampled from each bin.

2.2 Feature Combination Count (FCC) Encoding

Encoding density is a critical factor in predictor performance. Statistical encodings (like mean or standard deviation of features) are often too dense and lose structural information, while one-hot encodings result in massive, sparse vectors that are difficult for small predictors to process. ESM introduces FCC Encoding, which counts the occurrences of specific combinations of features (e.g., a specific kernel size paired with a specific expansion ratio). This captures vital inter-layer dependencies without the dimensionality explosion of one-hot vectors.

2.3 Iterative "Train-Evaluate-Extend" Loop

To maximize data efficiency, ESM employs an iterative strategy. The framework begins with a minimal dataset and only acquires additional hardware latency samples if the predictor fails to meet a predefined accuracy threshold. This "early exit" strategy ensures that researchers do not spend unnecessary time benchmarking on physical hardware once the surrogate model is sufficiently accurate.

3. Experimental Setup

The authors evaluated ESM across three diverse "supernet" search spaces:

  • ResNet-50 Supernet: $8.38 \times 10^{26}$ configurations.
  • MobileNetV3 Supernet: $8.38 \times 10^{26}$ configurations.
  • DenseNet Supernet: $10^{10}$ configurations.

Latency measurements were benchmarked across a wide spectrum of hardware, including high-end desktop GPUs (NVIDIA RTX 4090, RTX 3080 Max-Q), desktop CPUs (AMD Ryzen Threadripper 5975), and edge devices (Raspberry Pi 4). Accuracy was validated using the ImageNet dataset.

4. Key Results and Insights

4.1 Superior Predictive Accuracy

The FCC-based MLP predictor achieved an impressive 97.6% to 99% accuracy across all tested supernets. This significantly outperformed state-of-the-art statistical encodings, which typically achieved only 83-85% accuracy for complex spaces like ResNet.

4.2 Drastic Reductions in Data Requirements

The combination of balanced sampling and FCC encoding proved highly data-efficient. ESM reached its accuracy threshold with only 500 samples, whereas traditional random sampling required 4,000 samples to achieve comparable performance. This 8x reduction in required training data directly translates to an 8x speedup in the hardware-profiling stage of NAS.

4.3 Cross-Platform Robustness

One of the framework's strongest attributes is its consistency. Whether predicting for a high-throughput RTX 4090 or a resource-constrained Raspberry Pi, ESM maintained high accuracy. This suggests that FCC encoding effectively captures the structural nuances that influence latency across different hardware architectures and memory hierarchies.

5. Discussion and Limitations

The research highlights that the primary bottleneck in HW-NAS is data acquisition, not model training. By focusing on the density of the encoding and the distribution of the training samples, ESM provides a more practical path to hardware-aware optimization.

However, some limitations remain. The current framework is optimized for layer-wise or block-wise search spaces with a fixed macro-architecture. Its effectiveness in "cell-based" NAS, where global topology changes more drastically, remains to be explored. Additionally, while the framework is extensible to energy consumption, the current experimental focus is primarily on latency.

6. Conclusion

ESM represents a significant step forward in making Hardware-Aware NAS accessible. By addressing the bias of randomness in sampling and finding the "Goldilocks" zone of encoding density with FCC, the framework allows researchers to build highly accurate latency predictors with a fraction of the usual hardware-profiling effort.

Summary updated

BibTeX
@inproceedings{nasir2025esm,
  author = {Azaz-Ur-Rehman Nasir and Samroz Ahmad Shoaib and Muhammad Abdullah Hanif and Muhammad Shafique},
  title = {{ESM: A Framework for Building Effective Surrogate Models for Hardware-Aware Neural Architecture Search}},
  booktitle = {DAC},
  year = {2025},
  doi = {10.1109/dac63849.2025.11132572},
}
Figure