Introduction
Deploying machine learning models on "the Edge"—such as microcontrollers (MCUs) in smart speakers or wearables—presents significant challenges due to extremely limited memory and processing power. To address these constraints, researchers employ compression techniques like pruning (removing unnecessary weights), clustering (grouping similar weights), and quantization (reducing weight precision). While these techniques are effective individually, "collaborative optimization" chains them into a multi-stage pipeline. This study investigates whether the increased training overhead of complex pipelines is justified for small, dense models like those used in keyword spotting (KWS).
Methodology
The study utilizes the DS-CNN-S (Depthwise Separable Convolutional Neural Network - Small) architecture as a case study. Four optimization pathways were evaluated using the TensorFlow Model Optimization Toolkit (TF-MOT):
- QAT (Quantization-Aware Training): A baseline approach reducing weights to INT8.
- PQAT (Pruning + QAT): Pruning weights to specific sparsity levels before quantization.
- CQAT (Clustering + QAT): Grouping weights into clusters (8, 16, or 32) before quantization.
- PCQAT (Pruning + Clustering + QAT): A full collaborative pipeline involving all three techniques.
Performance metrics, including model size and inference latency, were estimated using Ethos-U Vela for the Arm Ethos-U55 microNPU.
Dataset and Infrastructure
The model was trained on the Google SpeechCommands v0.0.2 dataset, which contains one-second audio clips of 12 distinct keyword labels. Audio data is pre-processed into Mel Frequency Cepstral Coefficients (MFCCs), serving as visual "fingerprints" for the neural network.
To facilitate rapid experimentation, the training infrastructure was optimized by moving MFCC computation to the GPU and implementing parallel pre-fetching. These improvements reduced training time by 3x, from approximately 25 minutes to 8.5 minutes.
Results and Discussion
The experimental results reveal a clear trade-off between optimization complexity and hardware performance.
The Law of Diminishing Returns
While the full PCQAT pipeline achieved the smallest model size (24.66 KB) and fastest inference (113.20 µs), the gains over the simpler PQAT pathway (25.18 KB, 113.42 µs) were marginal. Given that PCQAT requires significantly more training time, the added complexity may not be warranted for "small-dense" models like DS-CNN-S, which already possess high parameter efficiency.
Clustering and Sparsity
Interestingly, CQAT models occasionally outperformed the uncompressed baseline in accuracy, suggesting that weight clustering can serve as a form of regularization. Conversely, the study identified a "catastrophic sparsity" threshold; pruning beyond 50% sparsity led to a sharp decline in model accuracy for this specific architecture.
Conclusion
For small-scale Edge AI models, a full-scale collaborative optimization pipeline often yields diminishing returns. While PCQAT provides the absolute best performance, simpler pathways like PQAT or even basic QAT offer a more efficient balance between training overhead and deployment performance. Future research should explore whether larger, more redundant architectures (e.g., MobileNetV2) derive more substantial benefits from these complex optimization chains.
Summary updated