Introduction
Electro-acoustic music presents a unique challenge for musicological analysis. Unlike traditional Western music, it is composed directly from "sound material" rather than written notation, resulting in a lack of standard scores. Analysis typically involves identifying "sound objects"—discrete events perceived as unified wholes. However, this process is inherently subjective and complicated by polyphony, where overlapping sounds mask one another. This paper introduces an interactive system that leverages human expertise through an active-learning loop to accurately transcribe these complex audio environments.
The Challenge of Subjectivity and Polyphony
The technical difficulty in automating electro-acoustic analysis is twofold. First, there is no objective "ground truth"; different analysts may segment the same piece differently based on their specific focus. Second, traditional audio classification often fails in dense compositions where multiple sound objects occur simultaneously. To address these issues, the proposed system acts as an intelligent assistant, learning a user’s specific analytical viewpoint in real-time.
System Architecture
The system operates via a two-phase workflow: Initialisation and an Interactive Loop.
Initialisation
Initially, the audio is partitioned into "texture segments"—short fragments characterized by a homogeneous timbre. Users interactively adjust the segmentation level using a slider to match the granularity of their analysis. They then select representative segments to define the sound classes they wish to track.
The Interactive Loop
The core of the system is an active learning cycle designed to minimize manual labor while maximizing accuracy.
- Learning: Support Vector Machine (SVM) classifiers are trained on user-validated segments.
- Prediction: The system labels the remaining audio segments based on the current model.
- Active Selection: The system identifies the most ambiguous segments—those where the classifier has the lowest confidence—and presents them to the user.
- Feedback: The user validates or corrects these labels, providing new training data for the next iteration.
Interaction Strategies
The researchers evaluated two primary strategies for user interaction:
- Multi-pass (MP): The user focuses on identifying one sound class at a time.
- One-pass (OP): The user identifies all classes simultaneously. A specific variant, OP2, allows for the creation of "mixture classes" (e.g., Class A + Class B). This enables the classifier to learn the specific characteristics of overlapping sounds rather than forcing a polyphonic segment into a single category.
Performance and Results
To evaluate the system, the authors used a synthetic dataset of 100 pieces with varying levels of polyphony (1 to 5 simultaneous sounds). This allowed for a rigorous comparison against a known ground truth.
The results demonstrate that the system is highly effective, even in dense audio environments. At a polyphony level of 4, the OP2 strategy achieved a median F-measure of 0.85 within just 20 iterations. The active learning approach proved significantly more efficient than random sampling, reaching high accuracy levels with minimal user intervention. Furthermore, the OP2 strategy outperformed the standard one-pass (OP1) approach, validating the utility of explicit mixture classes for handling polyphony.
Conclusion and Future Work
This research highlights the necessity of "human-in-the-loop" systems for subjective musicological tasks. By combining automated classification with expert feedback, the system significantly reduces the manual effort required for transcription. Future developments will focus on refining the user interface for real-world musicological workflows and improving the integration between the initial clustering phase and the subsequent classification loop.
Summary updated