Abstract
Environmental sound classification (ESC) is often hindered by the scarcity and imbalance of labeled audio data. This paper introduces a novel data augmentation technique termed Shuffling and Mixing (SM). By randomizing the temporal order of audio segments and mixing segments from different recordings of the same class, the proposed method generates diverse training samples without introducing statistical bias or unnatural event densities. When combined with a specialized Convolutional Neural Network (CNN) architecture designed for spectral and temporal feature extraction, the SM technique significantly improves classification performance. The system achieved a macro-averaged F1 score of 89.95% on the DCASE 2018 Task 5 challenge, outperforming the baseline by over 5%.
Introduction
Deep learning models for environmental sound classification require substantial amounts of high-quality labeled data. However, datasets for sounds like "eating" or "vacuum cleaning" are typically small and imbalanced. Existing augmentation techniques, such as pitch shifting or "Mixup" (overlaying two sounds), have notable drawbacks. Standard Mixup often doubles the "event density," creating sounds that are busier than real-world recordings and potentially biasing the model. Furthermore, these methods often preserve the original temporal sequence, which may not be a defining characteristic for many environmental sound classes.
Methods
The researchers propose a two-step augmentation strategy called Shuffling and Mixing (SM):
- Segmentation: An audio clip is divided into smaller, equal-length segments (e.g., 2 seconds).
- Shuffling: The temporal order of these segments is randomized. This creates a new sequence that remains representative of the original sound class.
- Mixing: Segments are drawn from two different recordings of the same class. A mixing ratio determines the proportion of segments taken from each source.
This approach allows for the generation of "busier" or "quieter" variations while maintaining the overall average density of the training set.
Complementing this augmentation is a specialized CNN architecture. It utilizes initial layers that convolve across the frequency axis to capture local spectral patterns, followed by layers that convolve across the time axis to capture temporal dependencies.
Key Insights: Temporal Independence
A core insight of this work is that environmental sounds are often composed of discrete, unordered events. For instance, in an "eating" class, the sequence of a plate clinking and a fork dropping does not change the classification.
By shuffling segments, the model learns to focus on the presence of these events rather than their specific order, leading to greater robustness.
Results
The method was evaluated on two major benchmarks:
- DCASE 2018 Task 5: The system achieved a macro-averaged F1 score of 89.95%, a significant improvement over the 84.50% baseline. This performance was instrumental in winning the challenge.
- Urban Sound 8K: The SM technique proved competitive with established methods like pitch shifting and particularly improved accuracy for classes like "Street Music" and "Engine Idling."
Ablation studies confirmed that the performance gains were a result of both the new CNN architecture and the SM augmentation working in tandem.
Conclusion
The Shuffling and Mixing technique provides a robust way to augment environmental sound data by leveraging the temporal independence of sound events. While highly effective for many classes, future work may explore its applicability to complex acoustic scenes with inherent temporal progressions and optimize segment lengths to avoid artifacts.
Summary updated