Abstract
Tiny machine learning (TinyML) has attracted heightened attention for its ability to provide low-cost and instantaneous performance on edge devices. Particularly, the commonly used microcontroller unit (MCU) imposes extreme constraints on peak memory (SRAM) and storage (Flash). Existing TinyML methods often rely on a customized and hard-to-obtain inference libraries, as well as necessitate a time-consuming search for a deployable architecture using advanced Neural Architecture Search (NAS) algorithms. To solve these problems, we fully exploit the resources on MCU and deduce hardware-oriented guidelines for designing models under extreme MCU constraints. In detail, we delve into thorough information about the atom operators by collecting the runtime data of Flash, SRAM, and latency to build a dataset named AtomDB. Based on AtomDB, several critical operator guidelines are established to fully utilize limited Flash and SRAM, while minimizing latency. By transferring the guidelines to analyze blocks, we propose a hybrid pattern that organizes appropriate blocks at different network stages to form the AtomNet, a more hardware-oriented architecture, to handle the former SRAM bottleneck and the latter Flash bottleneck. Extensive experiments demonstrate the effectiveness of the exploitation of the hardware characteristics. Remarkably, AtomNet pioneeringly achieve 3.5% accuracy enhancement and more than 15% latency reduction on 320KB MCU using readily available official inference libraries for ImageNet tasks, surpassing the current state-of-the-art method.
Introduction
Microcontrollers (MCUs) are the backbone of the IoT revolution, but they are notoriously difficult to program for deep learning. Unlike GPUs, MCUs have "extreme constraints": often less than 512KB of RAM (SRAM) and only a few megabytes of storage (Flash). Existing "TinyML" approaches usually rely on Neural Architecture Search (NAS), which treats the hardware as a black box or uses coarse, network-level estimates. These methods often fail to fully exploit the hardware's potential or require specialized, hard-to-find inference libraries.
AtomNet addresses these challenges by shifting the focus from the whole network to individual "atoms"—the fused operators (like Conv+BN+ReLU) that actually run on the chip. By optimizing at this granular level, AtomNet achieves state-of-the-art accuracy on extremely resource-constrained devices, surpassing previous methods like MCUNetV2 while significantly reducing latency.
Methodology: A Bottom-Up Design Philosophy
The researchers introduced a bottom-up design philosophy consisting of three main components: AtomDB, hardware-oriented guidelines, and a hybrid block pattern.
AtomDB and Kernel Additivity
The core of the approach is AtomDB, a massive database built by profiling thousands of individual operators on actual MCU hardware (STM32 series). The team measured exact Flash usage, peak SRAM, and latency for every "atom" operator. A critical finding was Kernel Additivity: the total latency of a block is essentially the sum of its constituent operators. This allows for incredibly accurate performance predictions without needing to run the full network on hardware during every search iteration.
Hardware-Oriented Guidelines
Based on AtomDB, three "golden rules" were derived:
- G1 (Flash-Friendly): Depthwise (DW) convolutions are more storage-efficient than Pointwise (PW) ones. Standard convolutions should be avoided.
- G2 (SRAM-Friendly): Avoid element-wise operations (like Add or Multiply) immediately after high-resolution layers, as they double the memory cost of the feature map.
- G3 (Latency-Friendly): Pointwise layers have higher "computation efficiency" (MACs/ms). Allocating more compute budget here provides the best accuracy-per-millisecond trade-off.
Hybrid Block Pattern
The research revealed that network bottlenecks shift as data flows through the model. Early stages are SRAM-dominant due to large input resolutions, while later stages are Flash-dominant due to the high number of channels. AtomNet employs a hybrid architecture: using SEP (Separable) blocks early on to save memory and ConvNext-style blocks later to maximize storage efficiency and accuracy.
Experimental Results
AtomNet was evaluated on standard benchmarks including ImageNet, Visual Wake Words (VWW), and Pascal VOC. The results consistently show that AtomNet pushes the Pareto frontier of accuracy and latency on MCUs.
Key Performance Metrics
- ImageNet: On the STM32F412 (256KB SRAM), AtomNet achieved 66.7% Top-1 accuracy, a 1.5% improvement over MCUNetV2. On the more powerful STM32H743 (512KB SRAM), it reached 73.0% Top-1 accuracy.
- Efficiency: On a 320KB MCU, AtomNet was 15% faster than the previous state-of-the-art while maintaining higher accuracy.
- Visual Wake Words: The model achieved a peak accuracy of 95.0%, outperforming the baseline MCUNet.
Discussion and Future Work
The success of AtomNet highlights the importance of hardware-aware design at the operator level. By recognizing that bottlenecks move from SRAM to Flash, the hybrid architecture avoids the sub-optimality of uniform block structures.
While the principles of AtomNet are general, the current AtomDB is specific to the STM32 architecture. Future work includes expanding this database to other architectures like ESP32 or RISC-V. Additionally, while the current "zero-cost" NAS is efficient, combining these hardware-oriented insights with evolutionary search algorithms or exploring lower-bit quantization (e.g., INT4) could further push the limits of TinyML.
Summary updated