Abstract
This paper describes our initial efforts to build a large-scale speaker diarization (SD) and identification system on a recently digitized radio broadcast archive from the Netherlands which has more than 6500 audio tapes with 3000 hours of Frisian-Dutch speech recorded between 1950–2016. The employed large-scale diarization scheme involves two stages: (1) tape-level speaker diarization providing pseudo-speaker identities and (2) speaker linking to relate pseudo-speakers appearing in multiple tapes. Having access to the speaker models of several frequently appearing speakers from the previously collected FAME! speech corpus, we further perform speaker identification by linking these known speakers to the pseudo-speakers identified at the first stage. In this work, we present a recently created longitudinal and multilingual SD corpus designed for large-scale SD research and evaluate the performance of a new speaker linking system using x-vectors with PLDA to quantify cross-tape speaker similarity on this corpus. The performance of this speaker linking system is evaluated on a small subset of the archive which is manually annotated with speaker information. The speaker linking performance reported on this subset (53 hours) and the whole archive (3000 hours) is compared to quantify the impact of scaling up in the amount of speech data.
Introduction
Standard speaker diarization (SD) is typically applied to single recordings, such as meetings or news clips. However, processing massive longitudinal archives—like the Omrop Fryslân archive containing over 6,500 tapes spanning six decades—presents unique challenges. Computational complexity makes global clustering on 3,000 hours of audio intractable, while voice changes over decades and varying acoustic contexts complicate consistent speaker identification. This research addresses these issues by developing a system that bridges the gap between local tape-level diarization and global speaker linking.
Methodology
The proposed system utilizes a robust two-stage pipeline designed for scalability.
- Stage 1: Tape-Level Diarization: Initial diarization is performed on individual tapes using the LIUM toolkit, generating "pseudo-speaker" labels (e.g., "Speaker A") for each file.
- Stage 2: Speaker Linking: This stage extracts speaker embeddings—either traditional statistical i-vectors or deep-learning-based x-vectors—from the segments identified in Stage 1. Similarity between these embeddings is scored using Probabilistic Linear Discriminant Analysis (PLDA). Finally, Agglomerative Hierarchical Clustering (AHC) with complete-linkage groups these pseudo-speakers into global identities.
Experimental Setup
The system was evaluated using the FAME! SD Corpus, a longitudinal and multilingual (Frisian-Dutch) dataset of ~3,000 hours recorded between 1950 and 2016. A 53-hour subset (82 tapes) with manual annotations served as the primary evaluation set. Embedding extractors were trained on standard large-scale datasets including Switchboard and SRE.
Results and Discussion
The primary metric for evaluation is the Diarization Error Rate (DER). Results indicate that neural x-vector embeddings consistently outperform traditional i-vectors, demonstrating greater robustness to diverse acoustic conditions and longitudinal voice changes.
Key findings include:
- Performance Gains: On the test set, the baseline (no linking) yielded a DER of 28.7%. Implementing speaker linking with x-vectors reduced this to 22.3% on the small subset and 24.5% when scaled to the full 3,000-hour archive.
- Scaling Impact: Linking across the entire archive is significantly more challenging than within a small subset. The DER increases as the search space expands, though the system still provides a net benefit over local diarization.
- Clustering Strategy: The system was tuned to favor over-segmentation in Stage 1, as merging clusters of the same speaker is more manageable than separating mixed-speaker clusters.
Conclusion
This work demonstrates the effectiveness of a two-stage diarization and linking approach for massive radio archives. While x-vectors provide superior accuracy, they exhibit sensitivity to the increased noise of a large-scale search space. Future work will explore meta-embeddings to improve clustering stability and integrate the system into a full spoken document retrieval platform for the Frisian archives.
Summary updated