Abstract
Deep learning models for audio classification typically require vast amounts of manually labeled data. This paper adapts the "Mean Teacher" semi-supervised learning (SSL) framework to the audio domain, introducing specific perturbations like environmental noise and sample mixing. The results demonstrate that consistency-based regularization significantly reduces the labeling burden, achieving near-state-of-the-art performance with only a fraction of the original training labels.
Introduction
Building high-performing audio classification models is often bottlenecked by the cost and time required for manual labeling. While Semi-Supervised Learning (SSL) has seen great success in computer vision by leveraging unlabeled data, its application to audio is less straightforward. Common image perturbations, such as rotation or flipping, do not have direct equivalents in the acoustic domain. This research investigates how consistency-based SSL can be effectively tailored for audio by identifying robust, domain-specific perturbations.
Methods: The Mean Teacher Framework
The authors utilize the Mean Teacher architecture, which consists of two neural networks: a Student and a Teacher. The Student model learns from labeled data via standard classification loss. The Teacher model's weights are an Exponential Moving Average (EMA) of the Student's weights, providing a stable target for consistency regularization.
The core mechanism involves feeding both models the same unlabeled audio clip with different random perturbations. The Student is penalized via a consistency loss if its prediction deviates from the Teacher's. Key perturbations introduced include:
- Time and Frequency Shifts: Spectrogram translations horizontally or vertically.
- Environmental Noise: Mixing in real-world sounds (e.g., traffic, household noises) rather than simple white noise.
- Sample Mixing (Mixup): Creating hybrid samples by linearly combining two clips and requiring the model to produce consistent combined predictions.
Datasets and Experimental Setup
The framework was evaluated on two major benchmarks:
- Google Speech Commands: 30 spoken command classes (e.g., "Yes", "No", "Up").
- UrbanSound8K: 10 classes of urban environmental sounds (e.g., car horns, dogs barking).
Key Results and Discussion
The experiments highlight the efficiency of SSL in low-data regimes. On the Google Speech Commands dataset, the Mean Teacher model achieved 92.45% accuracy using only 600 labels (1% of the total), significantly outperforming the supervised baseline of 87.04%.
As shown in the table above, the type of noise used for perturbation is critical. While standard Gaussian noise provides minimal benefit, incorporating diverse urban noise (MT+urban noise) pushed the 600-label accuracy to 94.31%. This suggests that real-world noise helps the model better distinguish the primary signal from complex backgrounds.
Furthermore, the "25% Rule" observed in the study indicates that the Mean Teacher model with only a quarter of the labels can match the performance of a fully supervised model using 100% of the data. On the UrbanSound8K dataset, the SSL approach improved accuracy by nearly 9% over the supervised baseline in low-label scenarios.
Conclusion
Consistency-based regularization, when paired with domain-appropriate perturbations like environmental noise and mixup, is a powerful tool for audio classification. These methods allow models to develop robust decision boundaries and ignore irrelevant background signals. Future work may extend these techniques to more complex tasks such as sound event detection or automated speech recognition, potentially exploring non-linear methods for audio sample mixing.
Summary updated