Abstract
Recognizing everyday environmental sounds remains a significant challenge in machine learning due to the high diversity of audio events and the limited availability of labeled datasets. This paper introduces Hierarchical Prototypical Networks (HPNs), a few-shot learning (FSL) framework that incorporates structured audio taxonomies into the training process. By organizing the embedding space to reflect semantic relationships between sounds, HPNs achieve state-of-the-art performance on the ESC-50 dataset, demonstrating that hierarchical knowledge is a powerful prior for learning from minimal examples.
Introduction
From the hum of a refrigerator to the chirp of a cricket, everyday sounds are ubiquitous yet difficult for AI to categorize reliably. While Few-Shot Learning (FSL) has emerged as a solution for recognizing new classes with only a few labeled examples (the "N-way K-shot" problem), most FSL algorithms treat labels as independent, flat categories. However, sounds naturally exist within hierarchies—for instance, "Dogs" and "Cats" are both "Animals," while "Chainsaws" and "Engines" fall under "Exterior/urban noises."
Humans leverage this structural knowledge to generalize quickly. This paper bridges the gap by integrating these taxonomies directly into the learning objective, forcing the model to understand not just what a sound is, but where it fits in the broader family tree of audio.
Methodology: Hierarchical Prototypical Networks
The proposed HPN architecture builds upon the standard Prototypical Network by introducing a multi-level embedding strategy.
Feature Extraction and Prototypes
The model utilizes an 8-layer VGG-like convolutional neural network (CNN) to transform log Mel spectrograms into 256-dimensional embeddings. In a standard Prototypical Network, a "prototype" is defined as the mean vector of all support examples for a specific class. HPNs extend this by calculating prototypes at every level of the hierarchy:
- Bottom Level: Prototypes are calculated for specific child classes (e.g., "Pig").
- Higher Levels: These specific prototypes are aggregated to form "parent" prototypes (e.g., "Animals"), representing the centroid of a broader category.
Multi-Task Learning and Structural Loss
To enforce the hierarchy, the model is trained to solve classification tasks at every level simultaneously. A weighted multi-level cross-entropy loss—referred to as Structural Loss—is applied. This ensures that the embedding space is organized such that "sibling" classes are positioned closer to each other than to "distant cousins," effectively using the taxonomy as a regularizer.
Experimental Setup
The researchers evaluated HPN on the ESC-50 dataset, which consists of 2,000 five-second recordings across 50 classes. These classes are grouped into five major parent categories: Animals, Natural & water soundscapes, Human non-speech sounds, Interior/domestic sounds, and Exterior/urban noises.
A key finding in the experimental setup was the importance of the data split. The researchers found that a "Uniform Split"—where the ratio of child classes under each parent remains consistent between training and testing—yielded the best results. This suggests that exposure to related "cousin" classes during training significantly aids the model's ability to generalize to new "sibling" classes during few-shot evaluation.
Results and Discussion
HPN consistently outperformed baseline few-shot methods, including standard Prototypical Networks, Matching Networks, and Transfer Learning approaches.
Performance Benchmarks
In the challenging 12-way 5-shot classification task, HPN achieved an accuracy of 78.65%, a notable improvement over the 77.70% achieved by standard Prototypical Networks. Similar gains were observed in 5-way 5-shot tasks and across F1-scores, indicating a robust balance between precision and recall.
The results confirm that a priori taxonomic knowledge acts as a powerful guide for the encoder, helping it learn embeddings that reflect semantic similarity rather than just raw acoustic features.
Conclusion and Future Work
While HPNs show great promise, they currently rely on predefined, explicit taxonomies and tree-based structures. Future work aims to address more complex "multi-path" taxonomies where a sound might belong to multiple categories. Additionally, the researchers plan to scale this approach to larger datasets like AudioSet and explore "knowledge-discovery" methods that can automatically infer hierarchical relationships without manual labeling.
The source code and evaluation environment are available on GitHub.
Summary updated