Prof-life-log: audio environment detection for naturalistic audio streams

Ali Ziaei, Abhijeet Sangwan, John H. L. HansenInterspeech 2012

TL;DR

This paper introduces "Prof-Life-Log," a novel system for unsupervised audio environment detection in naturalistic, everyday audio streams. By developing "acoustic signature vectors" (ASVs), the system can effectively match query audio segments to similar environments within a large collection, outperforming traditional GMM-UBM methods.

What Problem Does This Paper Address?

The increasing prevalence of personal audio recording devices (like smartphones) and the feasibility of continuous, long-duration audio capture present an opportunity for rich personal data analysis. A key challenge is making sense of these vast audio archives. Specifically, the ability to automatically detect and categorize the acoustic environment in which audio was recorded is crucial for applications like searching personal audio histories or generating summaries. Traditional methods often rely on supervised learning with pre-defined environments, which struggle with the dynamic and open-set nature of real-world audio, where environments can change rapidly and new ones emerge. This paper addresses the need for an unsupervised, efficient, and versatile system capable of identifying acoustic environments in naturalistic audio streams, even those not seen during training.

Methods

The proposed system, "Prof-Life-Log," tackles audio environment detection through a query-by-example approach. The core innovation lies in the creation of Acoustic Signature Vectors (ASVs).

Here's a breakdown of the method:

  1. Background Acoustic Model Training: A Gaussian Mixture Model (GMM) is trained on a large, diverse collection of unlabeled audio data. This GMM acts as a universal background model (UBM), similar to techniques used in speaker or language identification.
  2. ASV Generation: For each audio segment (e.g., 10-second chunks), the system computes the posterior probability of that segment's acoustic features being generated by each mixture in the trained GMM. This results in a vector of probabilities, the ASV, which is designed to capture the unique acoustic characteristics of the environment. Environments with distinct acoustic properties will have different ASV profiles.
  3. ASV Comparison (Query-by-Example): To find similar environments, a user provides a query audio segment. This segment is converted into its ASV. The similarity between the query ASV and the ASVs of all segments in the dataset is then computed using the log-cosine distance measure. Segments with a lower log-cosine distance are considered more similar to the query environment.

This unsupervised approach allows the system to handle novel environments without prior training data for them.

Datasets

The paper utilizes two types of datasets derived from the Prof-Life-Log corpus:

  • Homogenous Dataset: This dataset consists of recordings of specific, pure environments (e.g., restaurant, walking, computing cluster, street, music). These recordings are designed to be free of speech and represent distinct acoustic conditions. For evaluation, 600 segments were used for each of 5 environments, totaling 3600 segments.
  • Naturalistic Dataset: This dataset comprises real-world, long-duration audio recordings captured using the LENA unit. These recordings often contain mixed environments and speech. For evaluation, 1800 segments were used, representing 9 different environments (babble, clapping, indoors, laughing, typing, walking, wind, restaurant, outdoors).

Key Insights and Ideas

  • Acoustic Signatures for Environment Identity: The core idea that a vector of GMM posterior probabilities (ASV) can effectively represent and distinguish between diverse acoustic environments is a key insight. This moves beyond simple feature extraction to a more holistic acoustic fingerprint.
  • Unsupervised Learning for Real-World Robustness: The unsupervised nature of the ASV generation and matching is critical for real-world applications where pre-defined labels are impractical. The system can adapt to and identify environments it hasn't explicitly been trained on.
  • Query-by-Example Versatility: The query-by-example paradigm makes the system highly flexible. Users can pinpoint an environment of interest by providing just one example, and the system can then find all similar instances within a large collection.
  • Prof-Life-Log Corpus Value: The paper highlights the value of the Prof-Life-Log corpus as a unique resource for studying naturalistic audio environments, offering a contrast to more controlled, laboratory-based datasets.

Key Results

  • Homogenous Dataset Performance: The proposed ASV system achieved a significantly better Equal Error Rate (EER) than a traditional GMM-UBM system. For instance, with 128 GMM mixtures, the ASV system achieved an EER of 7.32%, compared to 16.38% for GMM-UBM. The ASV system's performance improved with more GMM mixtures, indicating better discrimination.
  • Naturalistic Dataset Performance: On the more challenging naturalistic dataset, the ASV system also outperformed GMM-UBM. The best performance was observed for the restaurant noise environment (~10% EER), while the office environment showed the worst performance (~45% EER), likely due to its quiet nature and less distinct acoustic events. The average EER across all 9 naturalistic environments was 22%.
  • Efficiency: The paper emphasizes the system's design for speed and efficiency, crucial for processing large audio collections.

Limitations and Future Work

  • Office Environment Difficulty: The paper notes that the office environment was particularly challenging to classify, likely due to its dominance of silence and less distinct acoustic events compared to environments like restaurants.
  • Short Segment Adaptation: The GMM-UBM system's performance was relatively constant, possibly because 10-second segments were insufficient for effective adaptation. The ASV system, however, benefited from increased model complexity.
  • Future Work: The authors suggest exploring more sophisticated acoustic models like Hidden Markov Models (HMMs) and investigating techniques similar to keyword spotting for acoustic environment detection.

Summary updated

BibTeX
@inproceedings{ziaei2012prof,
  author = {Ali Ziaei and Abhijeet Sangwan and John H. L. Hansen},
  title = {{Prof-life-log: audio environment detection for naturalistic audio streams}},
  booktitle = {Interspeech},
  year = {2012},
}
Figure