Abstract
Speaker diarization, the task of identifying "who spoke when," is critical for real-time audio applications such as automated transcription and meeting analysis. This paper presents a novel online speaker diarization system that utilizes a Size-Monitored Growing Neural Gas (GNG-T) algorithm. Unlike traditional offline methods that require processing the entire audio file, the proposed system segments and clusters speakers dynamically without prior knowledge of the speaker count or acoustic profiles. Evaluation on the BREF80 database demonstrates high accuracy in both turn detection and speaker identification across long-duration audio streams, maintaining stability even in 10-hour recordings.
Introduction
Speaker diarization serves as a foundational component for various audio processing tasks. While offline systems achieve high accuracy by processing entire audio files to optimize clustering, real-time or "online" diarization faces significant hurdles. Specifically, systems must handle an unknown number of speakers and adapt to non-stationary acoustic data where a speaker's voice profile may evolve over time. This research addresses these challenges by introducing a two-phase pipeline capable of real-time processing without "looking ahead" at future audio segments.
Methodology
Speaker Turn Detection
The initial phase involves segmenting the audio stream to identify speaker transitions. The system extracts Mel Frequency Cepstrum Coefficients (MFCCs) every 10ms to represent vocal textures. A sliding window approach, utilizing two 3-second windows, monitors the audio. By calculating the Mahalanobis distance between the acoustic vector distributions in these windows, the system identifies peaks that signify a change in speaker. This statistical measure allows the system to detect shifts in the acoustic space that correspond to speaker changes while ignoring minor intra-speaker variations.
Speaker Clustering via GNG-T
The core of the system is the Size-Monitored Growing Neural Gas (GNG-T) algorithm. Traditional clustering methods like k-means require a predefined number of clusters, which is impractical for online scenarios. In contrast, GNG-T offers:
- Dynamic Topology: It constructs a graph of nodes that grows to represent the distribution of a speaker's voice.
- Size-Monitoring: The algorithm adjusts the number of nodes to maintain a constant representation error, scaling the graph complexity based on the speaker's vocal characteristics. This ensures computational efficiency by reducing thousands of acoustic vectors into a small, representative graph.
- Graph Matching: New segments are compared against stored GNG-T graphs using a distance metric. If no match is found, a new speaker identity is initialized.
Experimental Setup
The system was evaluated using the BREF80 database, a corpus of French read-speech. The researchers selected 11 speakers (5 female, 6 male) and synthesized two test streams: a 1-hour file and a 10-hour file. These streams consisted of concatenated segments ranging from 10 to 40 seconds in length, designed to test the system's stability over long durations.
Results
Speaker Turn Detection Performance
The segmentation phase demonstrated high reliability across both test sets. As shown in Table 1, the system achieved a sensibility of 96.0% for the 1-hour database and 96.7% for the 10-hour database, indicating robust transition detection regardless of stream length.
Speaker Clustering Performance
The clustering results highlight the stability of the GNG-T approach. For the 1-hour stream, the system correctly identified 12 speakers (near the actual 11) with a matching accuracy of 97.0%. Performance remained consistent on the 10-hour stream with 95.1% accuracy. The purity metrics, as detailed in Table 2, further validate the clustering quality, with values near 1.0 indicating that the generated clusters are highly homogeneous.
Discussion and Conclusion
The GNG-T algorithm provides a computationally efficient "voice signature" through topological graphs, enabling real-time diarization. The system's ability to maintain high accuracy over a 10-hour stream proves it does not suffer from significant drift. However, limitations remain: the 3-second window requirement may lead to missed detections for very short utterances, and the current model does not account for overlapping speech. Future work will focus on refining the algorithm to handle rapid speaker changes and overlapping audio segments common in natural conversation.
Summary updated