Abstract
In this paper, we propose TitaNet, a novel neural network architecture for extracting speaker representations. We employ 1D depth-wise separable convolutions with Squeeze-and-Excitation (SE) layers with global context followed by channel attention based statistics pooling layer to map variable-length utterances to a fixed-length embedding (t-vector). TitaNet is a scalable architecture and achieves state-of-the-art performance on speaker verification task with an equal error rate (EER) of 0.68% on the VoxCeleb1 trial file and also on speaker diarization tasks with diarization error rate (DER) of 1.73% on AMI-MixHeadset, 1.99% on AMI-Lapel and 1.11% on CH109. Furthermore, we investigate various sizes of TitaNet and present a light TitaNet-S model with only 6M parameters that achieve near state-of-the-art results in diarization tasks.
Introduction
Speaker representation involves mapping variable-length audio signals into fixed-length mathematical vectors, commonly referred to as embeddings or "t-vectors." These representations are fundamental to two primary tasks: Speaker Verification (confirming a speaker's identity) and Speaker Diarization (determining "who spoke when").
Traditional models, such as the x-vector based on Time Delay Neural Networks (TDNN), have long served as industry standards. However, these architectures often struggle to capture long-range global context across an entire utterance. Additionally, many existing systems rely on complex multi-stage pipelines, often requiring external models like Probabilistic Linear Discriminant Analysis (PLDA) for scoring. TitaNet addresses these challenges by providing a scalable, end-to-end architecture that efficiently captures both local acoustic features and global speaker traits.
Architecture and Methods
TitaNet adapts the ContextNet architecture, originally designed for Automatic Speech Recognition (ASR), for the specific requirements of speaker recognition. The architecture is built upon three core components:
- 1D Depth-wise Separable Convolutions: By replacing standard convolutions with depth-wise separable versions, TitaNet significantly reduces parameter count and computational overhead while effectively modeling temporal patterns.
- Global Context via Squeeze-and-Excitation (SE) Layers: The model integrates SE modules that utilize global average pooling. This allows the network to incorporate information from the entire sequence to "re-weight" local features based on the global context of the utterance.
- Attentive Statistics Pooling: To aggregate frame-level features into a single speaker embedding, TitaNet employs a channel-attention mechanism. This ensures the model focuses on the most discriminative segments of the audio for identifying the speaker.
The model is trained using Additive Angular Margin (AAM) loss, which optimizes the embedding space such that embeddings from the same speaker are clustered closely on a hypersphere. This enables the use of simple cosine similarity for verification, eliminating the need for complex backends like PLDA.
Experimental Setup
TitaNet was trained on a comprehensive dataset comprising approximately 4.8 million utterances from over 16,000 speakers. This training set includes VoxCeleb1 & 2, NIST SRE, Fisher, Switchboard, and LibriSpeech, providing a diverse range of acoustic environments from "in-the-wild" recordings to telephonic speech. Evaluation was conducted on the VoxCeleb1 cleaned test trial for verification and benchmarks such as AMI (Lapel and MixHeadset), NIST-SRE-2000, and CH109 for diarization.
Key Results
TitaNet is offered in three scalable sizes: Small (6.4M parameters), Medium (13.4M), and Large (25.3M).
As shown in the table below, TitaNet-L achieves a state-of-the-art Equal Error Rate (EER) of 0.68% on VoxCeleb1, surpassing the performance of ECAPA-TDNN and Conformer-based models. Notably, TitaNet-S provides a highly efficient alternative, achieving competitive results with only 6.4 million parameters, making it suitable for edge deployments.
In diarization tasks, TitaNet-M achieved a Diarization Error Rate (DER) of 1.99% on the AMI-Lapel dataset, outperforming ECAPA-TDNN (2.36%) while maintaining a smaller parameter footprint.
Conclusion
TitaNet demonstrates that ASR-optimized architectures can be effectively adapted for speaker identity tasks by incorporating global context. The use of 1D depth-wise separable convolutions and SE layers provides a scalable and efficient path to state-of-the-art performance. While larger models yield significant gains in verification, the researchers observed diminishing returns in diarization once a sufficient level of embedding separability is reached. Future work will focus on performance in extremely noisy environments and overlapping speech scenarios.
Summary updated