Fast Data Aware Neural Architecture Search via Supernet Accelerated Evaluation

Emil Njor, Colby Banbury, Xenofon Fafoutis

TL;DR

TinyML deployment often fails because engineers optimize neural network architectures while leaving data configurations (like resolution) fixed. This paper introduces a Supernet-based Data Aware Neural Architecture Search (DA-NAS) that jointly optimizes both, discovering superior models that fit on microcontrollers twice as fast as previous methods.

What Problem Does This Paper Address?

Deploying machine learning on "Tiny" hardware—like the microcontrollers in your smart thermostat or wearable—is a nightmare of constraints. These devices have tiny amounts of RAM and Flash memory, requiring extreme optimization.

While Hardware-Aware Neural Architecture Search (HA-NAS) has automated the process of finding the right model structure, it usually treats the input data as a "given." However, in TinyML, the data configuration (e.g., image resolution or color depth) is just as critical as the model itself. A lower-resolution image uses less RAM, which might allow for a deeper, more accurate model within the same hardware budget. Manually balancing these trade-offs is slow and requires deep expertise. The authors aim to automate this joint "Data + Model" search to make TinyML more accessible and efficient.

Methods

The authors propose a Data Aware NAS that searches a joint space of data configurations and model architectures.

  1. Joint Search Space: The algorithm chooses from two data dimensions—resolution (from 32x32 to 224x224) and color format (RGB vs. Monochrome)—alongside model dimensions like depth (number of blocks) and width (alpha parameter) of a MobileNetV2 backbone.
  2. Supernet Acceleration: To avoid the massive computational cost of training every candidate model from scratch, they use a Supernet. They train a single, large "parent" model for each data configuration. During the search, they simply "sample" sub-networks from this parent. This "lazy training" approach provides pre-trained weights, requiring only minimal fine-tuning to evaluate a candidate.
  3. Genetic Search Strategy: They use a tournament-based genetic algorithm to evolve a population of TinyML systems, guided by a fitness function that penalizes models exceeding RAM or Flash limits while rewarding accuracy.
The conceptual shift from traditional HA-NAS to the proposed Data Aware NAS.

Datasets

  • Wake Vision: A large-scale, diverse dataset for person detection. This is the primary benchmark used to show the effectiveness of the supernet approach on complex, real-world tasks.
  • ToyADMOS: A simpler dataset for anomalous sound detection, used to revisit and validate findings from earlier, more "naive" versions of Data Aware NAS.

Key Insights and Ideas

  • The Data-Model Interplay: The most important takeaway is that data and architecture are not independent. By allowing the search to lower image resolution, the algorithm often finds it can fit a much more sophisticated model that ultimately achieves higher accuracy than a simple model processing high-res data.
  • Efficiency is a Choice: DA-NAS doesn't just find better models; it finds them faster. By exploring a more holistic search space, the algorithm reaches performance thresholds (like 67% accuracy) in roughly half the number of evaluations compared to methods that don't use the supernet acceleration.
  • Automated Adaptation: The system is "set and forget." An engineer only needs to change the hardware constraints (e.g., "I have 256KB RAM"), and the DA-NAS automatically re-balances the data resolution and model depth to fit, eliminating hours of manual trial-and-error.

Key Results

  • Higher Accuracy: On a standard TinyML hardware target (512 KB RAM / 2 MB Flash), DA-NAS found models with 79.5% accuracy, outperforming traditional HA-NAS which peaked at 77.6%.
  • More Options: DA-NAS discovered 22 models above the 75% accuracy threshold, whereas traditional methods only found 4. This gives engineers a much richer "Pareto frontier" of choices.
  • Speedup: The supernet-based implementation reached target accuracy levels in half the number of system evaluations compared to the authors' previous naive implementation.
  • Generalization: The method successfully scaled across "Large," "Medium," and "Small" hardware profiles, maintaining high performance even as RAM constraints were cut in half.

Limitations and Future Work

  • Proxy Metrics: The current implementation uses RAM and Flash consumption as proxies for hardware efficiency. The authors acknowledge that latency and energy consumption are harder to simulate accurately without deploying to real hardware, which remains a challenge for fast NAS.
  • Differentiable Search: While the genetic algorithm is effective, the authors suggest that moving toward Differentiable NAS (DNAS)—where the search space is continuous—could further accelerate the discovery of state-of-the-art TinyML systems.
  • Domain Expansion: The current work focuses heavily on vision. Future research could investigate how these data-model trade-offs manifest in other domains like audio or vibration sensing.

Summary updated

Abstract

Tiny machine learning (TinyML) promises to revolutionize fields such as healthcare, environmental monitoring, and industrial maintenance by running machine learning models on low-power embedded systems. However, the complex optimizations required for successful TinyML deployment continue to impede its widespread adoption. A promising route to simplifying TinyML is through automatic machine learning (AutoML), which can distill elaborate optimization workflows into accessible key decisions. Notably, Hardware Aware Neural Architecture Searches - where a computer searches for an optimal TinyML model based on predictive performance and hardware metrics - have gained significant traction, producing some of today's most widely used TinyML models. Nevertheless, limiting optimization solely to neural network architectures can prove insufficient. Because TinyML systems must operate under extremely tight resource constraints, the choice of input data configuration, such as resolution or sampling rate, also profoundly impacts overall system efficiency. Achieving truly optimal TinyML systems thus requires jointly tuning both input data and model architecture. Despite its importance, this "Data Aware Neural Architecture Search" remains underexplored. To address this gap, we propose a new state-of-the-art Data Aware Neural Architecture Search technique and demonstrate its effectiveness on the novel TinyML ``Wake Vision'' dataset. Our experiments show that across varying time and hardware constraints, Data Aware Neural Architecture Search consistently discovers superior TinyML systems compared to purely architecture-focused methods, underscoring the critical role of data-aware optimization in advancing TinyML.
BibTeX
@inproceedings{njor2025fast,
  author = {Emil Njor and Colby Banbury and Xenofon Fafoutis},
  title = {{Fast Data Aware Neural Architecture Search via Supernet Accelerated Evaluation}},
  booktitle = {Internet Things},
  year = {2025},
}
Figure