Weight-sharing Supernet for Searching Specialized Acoustic Event Classification Networks Across Device Constraints

Guan-Ting Lin, Qingming Tang, Chieh-Chi Kao, Viktor Rozgic, Chao Wang

Abstract

Acoustic Event Classification (AEC) is a critical component in modern smart devices, enabling features like home safety monitoring and accessibility support. However, deploying AEC models across a diverse landscape of hardware—from high-performance cloud servers to resource-constrained edge devices—presents a significant engineering challenge. Traditionally, optimizing for each hardware constraint requires training multiple independent models, a process that is computationally expensive and difficult to scale. In this paper, we present a Once-For-All (OFA) Neural Architecture Search (NAS) framework for AEC. By training a single weight-sharing supernet, we can extract specialized sub-networks tailored to specific computational budgets without retraining. Our results demonstrate that this approach achieves a 25.4% relative improvement over models trained from scratch and a 7.3% improvement over standard knowledge distillation, while revealing that the benefits of supernet training extend beyond architecture selection to fundamental model optimization.

Introduction

Acoustic Event Classification (AEC) allows devices to identify sounds such as smoke alarms, barking dogs, or breaking glass. As these capabilities move from the cloud to the "edge," developers face a dilemma: hardware fragmentation. A model optimized for a flagship smartphone may be too heavy for a budget smart speaker, while a model designed for a low-power sensor may underperform on more capable hardware.

Historically, this required an $O(N)$ approach, where $N$ different models were trained for $N$ different device constraints. This paper introduces OFA-AEC, a framework that reduces this complexity to $O(1)$. By training one "supernet" that contains many possible sub-architectures, we can instantly deploy optimized models across the entire spectrum of device constraints.

The OFA-AEC Framework

1. The Elastic Supernet

The core of the framework is a CNN-based supernet designed with "elasticity" in mind. Unlike static models, the supernet supports multiple architectural configurations through:

  • Elastic Width (EW): The number of channels in each layer can be dynamically adjusted.
  • Elastic Depth (ED): The number of layers within specific blocks can be varied (e.g., choosing between 1 or 2 layers per block).

2. Weight-Sharing and Training

Instead of training sub-networks independently, all possible architectures share weights within the master supernet. During training, different sub-networks are sampled and updated. This ensures that when a specific "slice" of the model is extracted for a device, the weights are already pre-optimized for that specific configuration.

3. Knowledge Distillation (KD)

To maintain high accuracy in smaller sub-networks, the authors employed a large, pre-trained "teacher" model (CNN10). The supernet is trained to mimic the teacher's output distribution, effectively transferring complex feature representations into the more efficient elastic blocks.

Once the supernet is trained, finding the optimal model for a specific device becomes a simple search task. Given a parameter budget (e.g., < 1 million parameters), a search algorithm identifies the sub-network configuration within the supernet that yields the highest performance on a validation set.

Experimental Setup

The researchers evaluated the framework using AudioSet, the industry-standard benchmark for audio classification. The training was conducted on the balanced subset (approx. 22k samples), and performance was measured on the official evaluation set (approx. 20k samples). Audio data was preprocessed into log-mel spectrograms with 64 bins and 1000 frames for each 10-second clip.

Key Insights

The "Early Layer" Stability Requirement

A significant finding was that making the initial layers of the network elastic actually degraded performance. The researchers discovered that the early stages of an audio model must remain stable to extract fundamental acoustic features. Consequently, elasticity was only applied to later stages of the network, where the model learns more abstract representations.

The study found that the supernet approach provides a "regularization effect." An ultra-small model extracted from the supernet consistently outperformed the exact same architecture when trained from scratch. This suggests that the weight-sharing process itself improves the optimization landscape, making the resulting models more robust than their independently trained counterparts.

Results

The OFA approach demonstrated superior performance across various parameter budgets:

  • Superior Accuracy: For ultra-small models (~0.9M parameters), OFA-AEC achieved 26.6 mAP. This represents a 25.4% relative improvement over baseline models and a 7.3% improvement over models using standard knowledge distillation.
  • High Efficiency: The searched models retained nearly 90% of the performance of the massive CNN10 teacher model while using only 17.3% of the parameters.
  • Scalability: A single training session successfully produced a family of high-performing models ranging from 0.8M to 3.8M parameters, covering a wide array of deployment scenarios.

Conclusion

The OFA-AEC framework provides a scalable solution to the problem of device fragmentation in audio sensing. By shifting the burden from repetitive training to a single supernet optimization, the framework allows for the rapid deployment of specialized models. Future work may explore applying these principles to Transformer-based architectures and expanding the search space to include other audio tasks such as keyword spotting and music classification.

Summary updated

Abstract

Acoustic Event Classification (AEC) has been widely used in devices such as smart speakers and mobile phones for home safety or accessibility support. As AEC models run on more and more devices with diverse computation resource constraints, it became increasingly expensive to develop models that are tuned to achieve optimal accuracy/computation trade-off for each given computation resource constraint. In this paper, we introduce a Once-For-All (OFA) Neural Architecture Search (NAS) framework for AEC. Specifically, we first train a weight-sharing supernet that supports different model architectures, followed by automatically searching for a model given specific computational resource constraints. Our experimental results showed that by just training once, the resulting model from NAS significantly outperforms both models trained individually from scratch and knowledge distillation (25.4% and 7.3% relative improvement). We also found that the benefit of weight-sharing supernet training of ultra-small models comes not only from searching but from optimization.
BibTeX
@inproceedings{lin2023weight,
  author = {Guan-Ting Lin and Qingming Tang and Chieh-Chi Kao and Viktor Rozgic and Chao Wang},
  title = {{Weight-sharing Supernet for Searching Specialized Acoustic Event Classification Networks Across Device Constraints}},
  booktitle = {IEEE International Conference on Acoustics, Speech and Signal Processing},
  year = {2023},
  doi = {10.1109/icassp49357.2023.10096117},
}
Figure