Acoustic Scene Classification Using Teacher-Student Learning with Soft-Labels

Hee-Soo Heo, Jee-weon Jung, Hye-jin Shim, Ha-Jin YuInterspeech 2019 · DOI

Introduction

Acoustic Scene Classification (ASC) aims to categorize audio recordings into predefined environmental classes, such as "Airport," "Park," or "Metro Station." Traditionally, this task is approached using supervised learning with one-hot labeling, where a model is trained to assign a 100% probability to the ground-truth class and 0% to all others.

However, this approach assumes that acoustic scenes are strictly orthogonal—completely unrelated to one another. In reality, many environments share significant acoustic properties. An airport and a shopping mall, for instance, both feature background "babble noise" from crowds and the sound of footsteps on hard floors. Forcing a model to treat these overlapping scenes as entirely distinct is inefficient and contradicts human perception. This paper proposes a Teacher-Student (TS) learning framework that utilizes "soft-labels" to capture these inter-class similarities, leading to improved generalization and classification accuracy.

Methodology: The Teacher-Student Framework

The core of the proposed method is a Teacher-Student architecture where a "Student" network is trained to mimic the behavior of a high-performing "Teacher" network. The researchers introduce three key innovations to enhance this distillation process:

Temporal Superiority

To ensure the Teacher is more capable than the Student, the authors provide it with more temporal context. While the Student processes standard 10-second audio clips, the Teacher is fed 20 or 30-second clips created by concatenating multiple segments from the same scene. This longer duration allows the Teacher to extract more stable and discriminative features.

t-SNE visualization of embeddings for long (30s) vs. short (10s) audio segments.

As shown in the t-SNE visualization above, embeddings from longer audio segments (30s) cluster much more tightly than those from short segments (10s). This tighter clustering confirms that increased temporal context is a primary driver of the Teacher's superior performance.

Embedding-Level Distillation

Rather than only distilling knowledge from the final 10-class probability output, the authors implement Embedding-Level Distillation. The Student is forced to match the Teacher’s last hidden layer embeddings using Mean Squared Error (MSE). The hypothesis is that these high-dimensional embeddings capture the "manifold" of acoustic similarities—the nuanced relationships between different scenes—more effectively than final probability scores.

Knowledge Distillation with Temperature

To further reveal these relationships, the researchers use a "temperature" variable ($T$) to soften the probability distributions. A higher temperature (found to be optimal at $T=5$) flattens the distribution, making the "soft-labels" more informative by highlighting which non-target classes the Teacher finds most similar to the ground truth.

Experimental Setup

The study utilized the DCASE 2018 Task 1-a dataset, consisting of 8,640 stereo audio segments (10 seconds each) across 10 scene categories. The system employed an ensemble of two front-end models: a CNN for raw waveforms and a CNN-GRU (Gated Recurrent Unit) for spectrograms, with an SVM back-end for final classification.

Performance and Confusion Reduction

The proposed Teacher-Student approach provided a significant performance boost over a strong baseline. While the baseline ensemble (without TS learning) achieved 74.42% accuracy, the proposed TS system reached 77.36% accuracy.

Beyond the overall accuracy jump, the system significantly reduced errors between the most "confusable" pairs of scenes.

Confusion matrices comparing the baseline model to the model trained with Teacher-Student learning.

As illustrated in the confusion matrices, the TS learning framework noticeably reduced the confusion between "shopping mall" and "airport," as well as between "metro" and "tram." By using soft-labels, the model learns that an airport should sound somewhat like a mall, preventing it from making rigid, incorrect guesses when faced with ambiguous audio.

Conclusion

By moving away from the "orthogonality myth" of one-hot labels, this research demonstrates that capturing inter-class similarities is vital for robust Acoustic Scene Classification. The Teacher-Student framework, bolstered by temporal superiority and embedding-level distillation, allows models to learn a richer, more relational representation of the acoustic world. Future work may explore more complex distance metrics for embedding distillation and the application of this method to datasets with scarcer labels.

Summary updated

BibTeX
@inproceedings{heo2019acoustic,
  author = {Hee-Soo Heo and Jee-weon Jung and Hye-jin Shim and Ha-Jin Yu},
  title = {{Acoustic Scene Classification Using Teacher-Student Learning with Soft-Labels}},
  booktitle = {Interspeech},
  year = {2019},
  doi = {10.21437/interspeech.2019-1989},
}
Figure