DiTMoS: Delving into Diverse Tiny-Model Selection on Microcontrollers

Xiao Ma, Shengfeng He, Hezhe Qiao, Dong Ma

TL;DR

This paper introduces DiTMoS, a novel framework for running accurate deep learning models on resource-constrained microcontrollers (MCUs). Instead of compressing large models, DiTMoS builds an ensemble of small, diverse "weak" models and uses a "selector" to route each input to the best-suited model, achieving significant accuracy improvements.

What Problem Does This Paper Address?

Deep neural networks (DNNs) offer powerful capabilities for analyzing sensor data, but deploying them on microcontrollers (MCUs) is a major challenge. MCUs have extremely limited memory and processing power, making it difficult to run even moderately sized DNNs. Current approaches often involve compressing larger, more accurate models, which can lead to a significant drop in performance. This paper tackles this by proposing a "bottom-up" approach: creating small, specialized models and intelligently combining their strengths to achieve high accuracy without the need for large, pre-trained models.

Methods

DiTMoS employs a unique "selector-classifiers" architecture. The core idea is that a collection of diverse, weak models, when intelligently selected, can collectively outperform a single, larger model.

Here are the key techniques:

  • Diverse Training Data Splitting: To ensure the weak models are truly diverse, the training data is split into multiple subsets. This is done by first using a pre-trained "strong" model to extract features and then clustering these features (using K-Means) to create distinct subsets. Each weak classifier is then trained on its specific subset, forcing it to learn specialized knowledge.
  • Adversarial Selector-Classifiers Training: The selector and classifiers are trained iteratively and asynchronously in an adversarial manner. This means one component is frozen while the other is trained, forcing them to learn to work together effectively. This "coupling" is crucial for the selector to accurately route inputs and for the classifiers to complement each other.
  • Heterogeneous Feature Aggregation: To boost the classifiers' performance, DiTMoS aggregates intermediate features from the selector's early layers. This allows the classifiers to benefit from the global, task-neutral features learned by the selector, enhancing their representation capability.
  • Network Slicing: To mitigate the extra memory overhead introduced by feature aggregation, a network slicing technique is used. This involves strategically storing intermediate activations in the MCU's Flash memory instead of the limited SRAM, effectively eliminating most of the memory increase at the cost of a slight latency increase.

Datasets

The paper evaluates DiTMoS on three public time-series datasets suitable for MCU deployment:

  • UniMiB-SHAR: Human activity recognition using accelerometer data.
  • Speech Commands: Keyword spotting from audio data.
  • DEAP: Emotion recognition from EEG signals.

These datasets were split into 80% training and 20% testing.

Key Insights and Ideas

  • The Power of Union Accuracy: The paper highlights that the "union accuracy" (the percentage of samples correctly classified by at least one model in an ensemble) is significantly higher for diverse weak models than for strong models. This suggests that a well-chosen ensemble of weak models can achieve a higher accuracy ceiling.
  • Diversity is Key: The limited capacity of weak models naturally leads to greater diversity in the features they learn, which is beneficial for ensemble performance.
  • Selector-Classifiers Synergy: Simply training a selector and classifiers independently doesn't work well. An adversarial, iterative training process is essential to create a strong coupling and ensure they work in concert.
  • Feature Aggregation for Weak Models: Even small models can benefit from leveraging intermediate features from a routing mechanism, especially when the models are inherently weak.
  • Network Slicing for Memory Constraints: The proposed network slicing technique is a clever way to manage memory on MCUs by intelligently using Flash, a common trade-off for memory-constrained systems.

Key Results

  • Accuracy Improvement: DiTMoS achieved up to 13.4% accuracy improvement compared to the best baseline across the evaluated datasets. Specifically, it reached 86.2% on UniMiB-SHAR, 86.2% on Speech Commands, and 77.4% on DEAP.
  • Memory Efficiency: With network slicing, DiTMoS almost completely eliminated the extra memory overhead incurred by feature aggregation, with only a marginal increase in latency (around 1.6 ms).
  • Adaptability to Model Size: DiTMoS consistently outperformed baselines across different model sizes, showing particular strength with smaller models where diversity is more pronounced.
  • Ablation Study: The paper demonstrated that each component of DiTMoS (diverse data splitting, adversarial training, and feature aggregation) contributes to the overall performance, with adversarial training being the most impactful.

Limitations and Future Work

  • Number of Classifiers Trade-off: While more classifiers can increase union accuracy, they also increase the difficulty for the selector and require more Flash storage. The paper suggests that application designers can choose the optimal number based on their specific needs and available resources.
  • Architecture Flexibility: The paper used the same architecture for the selector and classifiers for demonstration. Future work could explore designing these components with different architectures and kernel sizes to further enhance diversity and performance.
  • Generalizability of Slicing: While network slicing effectively addresses memory overhead, its impact on latency might be a concern for highly latency-sensitive applications. Further research could explore more optimized slicing strategies.

Summary updated

Abstract

Enabling efficient and accurate deep neural network (DNN) inference on microcontrollers is non-trivial due to the constrained on-chip resources. Current methodologies primarily focus on compressing larger models yet at the expense of model accuracy. In this paper, we rethink the problem from the inverse perspective by constructing small/weak models directly and improving their accuracy. Thus, we introduce DiTMoS, a novel DNN training and inference framework with a selector-classifiers architecture, where the selector routes each input sample to the appropriate classifier for classification. DiTMoS is grounded on a key insight: a composition of weak models can exhibit high diversity and the union of them can significantly boost the accuracy upper bound. To approach the upper bound, DiT-MoS introduces three strategies including diverse training data splitting to increase the classifiers' diversity, adversarial selector-classifiers training to ensure synergistic interactions thereby maximizing their complementarity, and heterogeneous feature aggregation to improve the capacity of classifiers. We further propose a network slicing technique to alleviate the extra memory overhead incurred by feature aggregation. We deploy DiTMoS on the Neucleo STM32F767ZI board and evaluate it based on three time-series datasets for human activity recognition, keywords spotting, and emotion recognition, respectively. The experiment results manifest that: (a) DiTMoS achieves up to 13.4% accuracy improvement compared to the best baseline; (b) network slicing almost completely eliminates the memory overhead incurred by feature aggregation with a marginal increase of latency. Code is released at https//github.com/TheMaXiao/DiTMoS
BibTeX
@inproceedings{ma2024ditmos,
  author = {Xiao Ma and Shengfeng He and Hezhe Qiao and Dong Ma},
  title = {{DiTMoS: Delving into Diverse Tiny-Model Selection on Microcontrollers}},
  booktitle = {PerCom},
  year = {2024},
  doi = {10.1109/percom59722.2024.10494422},
}
Figure