Timed Probabilistic Automaton: A Bridge between Raven and Song Scope for Automatic Species Recognition

AAAI 2013

TL;DR

This paper introduces Timed Probabilistic Automata (TPA) as a novel approach to bridge the gap between two existing bioacoustic analysis tools, Raven and Song Scope. TPA aims to improve automatic species recognition by jointly detecting individual sound components (syllables) and their hierarchical structures (call structures), outperforming the individual tools in challenging acoustic environments.

What Problem Does This Paper Address?

Environmental monitoring relies heavily on identifying animal vocalizations to assess biodiversity and ecosystem health. While manual analysis is feasible for small datasets, it's impractical for large-scale, long-term studies. Automated species recognition tools like Raven and Song Scope have emerged to tackle this challenge. However, Raven excels at detecting individual sound units (syllables) but struggles with recognizing complex call structures, while Song Scope can identify call structures but is sensitive to noise and syllable purity. This leaves a gap in accurately recognizing species calls, especially in complex, noisy environments like dawn choruses where multiple species vocalize simultaneously. The paper addresses this by proposing a unified approach that combines the strengths of both existing tools.

Methods

The core contribution is the Timed Probabilistic Automata (TPA) system. TPA is built upon Syntactic Pattern Recognition and Markov Models, adapting them for avian call structures. The process involves:

  • Acoustic Component Detectors: Specialized filters designed to identify five key acoustic components (lines, warbles, blocks, oscillations, stacked harmonics) within spectrograms. These detectors are parametric and configurable.
  • Component Sequence Derivation: Identified shapes in the spectrogram are translated into a sequence of acoustic components, each characterized by its shape, timing, duration, and frequency.
  • Timed Automata: A probabilistic timed finite automaton is used to model the sequence of components and their temporal relationships, effectively representing the "language" of a species' call.
  • Classification: A recognizer, trained on positive examples, uses the probabilistic automata to classify previously unlabelled input.

This approach differs from existing tools by explicitly separating the detection of individual components from the modeling of their hierarchical arrangement into call structures, and then integrating these two aspects.

Datasets

  • Training and Testing Dataset: Collected from Samford Valley, Queensland, Australia, during dawn choruses (4 am - 9 am) on October 14th, 2010. The dataset was tagged by birdwatching enthusiasts using a custom online acoustic analysis tool.
  • Species Selection: Five representative species were chosen to cover different call structures: Lewin's Honeyeater (oscillations), Eastern Whipbird (whistles and clicks), Eastern Koel (warbles), Torresian Crow (stacked harmonics), and Rainbow Lorikeet (blocks).
  • Data Volume: The dataset comprised 131 minutes for HoneyEater, 167 minutes for Eastern Whipbird, 237 minutes for Eastern Koel, 67 minutes for Torresian Crow, and 93 minutes for Rainbow Lorikeet.
  • Training Data: 25 samples per species were selected from a different day (October 15th, 2010) from the same site.

Key Insights and Ideas

  • The Syllable vs. Call Structure Dichotomy: Existing tools are specialized, either focusing on individual sound units (syllables) or the overall structure of calls, leading to limitations. TPA aims to bridge this by handling both.
  • Noise Sensitivity: Song Scope, relying on Hidden Markov Models, is particularly sensitive to noisy syllables, which can lead to misclassification. Raven's simpler detectors are less affected by noisy syllables but may have higher false positives.
  • Dawn Chorus Challenges: The dawn chorus is inherently difficult for automated recognition due to the high density of simultaneous vocalizations from multiple species, impacting both human and machine performance.
  • TPA's Unified Approach: By separating component detection and structure modeling, TPA offers a more robust framework that can potentially overcome the limitations of individual tools.

Key Results

  • TPA Outperforms Raven and Song Scope: For the Eastern Whipbird, TPA achieved significantly higher precision, recall, and accuracy compared to Raven and Song Scope.
  • TPA: Precision 0.45, Recall 0.59, Accuracy 0.52
    • Raven: Precision 0.28, Recall 0.41, Accuracy 0.35
    • Song Scope: Precision 0.29, Recall 0.38, Accuracy 0.33
  • Zero Precision Handling: TPA showed a lower percentage of "zero precision" instances (cases where no correct detections were made) compared to Raven and Song Scope, indicating better performance even in challenging noisy conditions.
  • Raven vs. Song Scope Performance: On average, Raven had higher recall (detecting more calls) but lower precision (more false positives), making it suitable for activity level detection. Song Scope had higher precision but lower recall, making it better for presence detection.

Limitations and Future Work

  • Accuracy in Noisy Environments: Despite improvements, TPA's accuracy (around 50% for the Eastern Whipbird) is still not at a level where ecologists can use it without careful verification.
  • Generalizability: The current research is focused on a specific species and environment. Future work needs to extend testing to multiple sites, days, and a wider variety of species with more complex call structures.
  • User Interface and Accessibility: While TPA is presented as a framework, its practical implementation and user-friendliness for ecologists are areas for further development.
  • Ongoing Research: The TPA approach is still under testing and construction, indicating that further refinement and validation are necessary.

Summary updated

BibTeX
@inproceedings{2013timed,
  title = {{Timed Probabilistic Automaton: A Bridge between Raven and Song Scope for Automatic Species Recognition}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2013},
}
Figure