TL;DR
TinyML deployment often fails because engineers optimize neural network architectures while leaving data configurations (like resolution) fixed. This paper introduces a Supernet-based Data Aware Neural Architecture Search (DA-NAS) that jointly optimizes both, discovering superior models that fit on microcontrollers twice as fast as previous methods.
What Problem Does This Paper Address?
Deploying machine learning on "Tiny" hardware—like the microcontrollers in your smart thermostat or wearable—is a nightmare of constraints. These devices have tiny amounts of RAM and Flash memory, requiring extreme optimization.
While Hardware-Aware Neural Architecture Search (HA-NAS) has automated the process of finding the right model structure, it usually treats the input data as a "given." However, in TinyML, the data configuration (e.g., image resolution or color depth) is just as critical as the model itself. A lower-resolution image uses less RAM, which might allow for a deeper, more accurate model within the same hardware budget. Manually balancing these trade-offs is slow and requires deep expertise. The authors aim to automate this joint "Data + Model" search to make TinyML more accessible and efficient.
Methods
The authors propose a Data Aware NAS that searches a joint space of data configurations and model architectures.
- Joint Search Space: The algorithm chooses from two data dimensions—resolution (from 32x32 to 224x224) and color format (RGB vs. Monochrome)—alongside model dimensions like depth (number of blocks) and width (alpha parameter) of a MobileNetV2 backbone.
- Supernet Acceleration: To avoid the massive computational cost of training every candidate model from scratch, they use a Supernet. They train a single, large "parent" model for each data configuration. During the search, they simply "sample" sub-networks from this parent. This "lazy training" approach provides pre-trained weights, requiring only minimal fine-tuning to evaluate a candidate.
- Genetic Search Strategy: They use a tournament-based genetic algorithm to evolve a population of TinyML systems, guided by a fitness function that penalizes models exceeding RAM or Flash limits while rewarding accuracy.
Datasets
- Wake Vision: A large-scale, diverse dataset for person detection. This is the primary benchmark used to show the effectiveness of the supernet approach on complex, real-world tasks.
- ToyADMOS: A simpler dataset for anomalous sound detection, used to revisit and validate findings from earlier, more "naive" versions of Data Aware NAS.
Key Insights and Ideas
- The Data-Model Interplay: The most important takeaway is that data and architecture are not independent. By allowing the search to lower image resolution, the algorithm often finds it can fit a much more sophisticated model that ultimately achieves higher accuracy than a simple model processing high-res data.
- Efficiency is a Choice: DA-NAS doesn't just find better models; it finds them faster. By exploring a more holistic search space, the algorithm reaches performance thresholds (like 67% accuracy) in roughly half the number of evaluations compared to methods that don't use the supernet acceleration.
- Automated Adaptation: The system is "set and forget." An engineer only needs to change the hardware constraints (e.g., "I have 256KB RAM"), and the DA-NAS automatically re-balances the data resolution and model depth to fit, eliminating hours of manual trial-and-error.
Key Results
- Higher Accuracy: On a standard TinyML hardware target (512 KB RAM / 2 MB Flash), DA-NAS found models with 79.5% accuracy, outperforming traditional HA-NAS which peaked at 77.6%.
- More Options: DA-NAS discovered 22 models above the 75% accuracy threshold, whereas traditional methods only found 4. This gives engineers a much richer "Pareto frontier" of choices.
- Speedup: The supernet-based implementation reached target accuracy levels in half the number of system evaluations compared to the authors' previous naive implementation.
- Generalization: The method successfully scaled across "Large," "Medium," and "Small" hardware profiles, maintaining high performance even as RAM constraints were cut in half.
Limitations and Future Work
- Proxy Metrics: The current implementation uses RAM and Flash consumption as proxies for hardware efficiency. The authors acknowledge that latency and energy consumption are harder to simulate accurately without deploying to real hardware, which remains a challenge for fast NAS.
- Differentiable Search: While the genetic algorithm is effective, the authors suggest that moving toward Differentiable NAS (DNAS)—where the search space is continuous—could further accelerate the discovery of state-of-the-art TinyML systems.
- Domain Expansion: The current work focuses heavily on vision. Future research could investigate how these data-model trade-offs manifest in other domains like audio or vibration sensing.
Summary updated