Improving Transformer-based End-to-End Speaker Diarization by Assigning Auxiliary Losses to Attention Heads

Ye-Rin Jeoung, Joon-Young Yang, Jeong-Hwan Choi, Joon-Hyuk Chang

TL;DR

This paper introduces a novel training strategy for Transformer-based End-to-End Neural Speaker Diarization (EEND) models by assigning auxiliary losses to specific self-attention heads. By identifying redundant "identity-like" attention heads and forcing them to learn speaker-wise voice activity and overlap detection patterns, the authors significantly reduce diarization error rates on both simulated and real conversational datasets.

What Problem Does This Paper Address?

Speaker diarization—solving the question of "who spoke when"—is critical for processing multi-speaker audio like meetings, interviews, and phone calls. Traditional systems use a pipeline of independent modules (voice activity detection, clustering, etc.), which are prone to compounding errors. End-to-End Neural Diarization (EEND) models solve this by directly predicting speaker activities using multi-head self-attention (MHSA) mechanisms.

However, standard self-attentive EEND (SA-EEND) models suffer from structural redundancy. As the network gets deeper, many attention heads default to "identity-like" patterns, meaning they primarily attend to the frame itself rather than capturing broader temporal or contextual relationships. This waste of model capacity limits performance, particularly in challenging scenarios like overlapping speech.

Methods

To combat this redundancy, the authors propose injecting auxiliary tasks directly into the underutilized attention heads during training.

The process follows three main steps:

  1. Identifying Redundant Heads: The authors calculate the mathematical trace of the attention weight matrices for each head. A high trace indicates that the head behaves like an identity matrix (focusing only on the diagonal). The heads with the highest traces are selected for regularization.
  2. Speaker-wise Voice Activity Detection (SVAD) Loss: Selected heads are forced to match a target mask derived from true individual speaker labels using Binary Cross-Entropy (BCE) loss. This guides the heads to learn distinct speaker activity patterns.
  3. Overlapped Speech Detection (OSD) Loss: Another head is constrained to match an OSD target mask using Mean Squared Error (MSE) loss, helping the model explicitly differentiate between non-speech, single-speaker, and overlapping speech regions.
Fig. 1. SA-EEND model with proposed auxiliary losses for two speakers

The final objective function combines the standard diarization loss ($L_d$) with the weighted auxiliary losses ($L_S$ and $L_O$).

Datasets

The approach was evaluated on two primary benchmarks:

  • Sim2spk: A simulated two-speaker dataset generated from Switchboard and NIST Speaker Recognition Evaluation corpora, featuring a high overlap ratio of 34.4% in training. Evaluation sets included varying overlap ratios (34.4%, 27.3%, and 19.6%).
  • CALLHOME (CH): A real-world dataset consisting of two-speaker telephone conversations, used for model adaptation (155 mixtures) and evaluation (148 mixtures).

Key Insights and Ideas

  • Targeted Regularization Beats Global Constraints: Instead of applying auxiliary losses to entire layers or adding complex residual connections (like previous architectures such as RX-EEND), targeting specific underperforming heads preserves the model's overall capacity while forcing diversification.
  • Layer Depth Matters for Sub-tasks: Through empirical testing, the authors found that the SVAD loss yields the best results when applied to the final (4th) encoder block, whereas the OSD loss is most effective when applied to the first encoder block. This suggests that overlap detection is a low-level feature best captured early, while speaker-specific tracking requires high-level abstractions found deeper in the network.
  • Visual Proof of Diversification: Visualizing the attention matrices confirmed that the proposed losses successfully broke up the identity-like diagonal lines, resulting in much richer, non-diagonal attention patterns that capture long-range speaker interactions.
Fig. 2. Visualization of attention weight matrices in the 4th encoder block: (a) SA-EEND, (b) RX-EEND, (c) SA-EEND with SVAD loss (d) SA-EEND with OSD loss, and (e) SA-EEND with both auxiliary losses applied to the 4th encoder block. (CH recording id: iaai)

Key Results

The proposed method achieved substantial performance gains over the baseline SA-EEND model across all test sets:

  • Simulated Data (Sim2spk): At the default 34.4% overlap ratio, the Diarization Error Rate (DER) dropped from 6.30% (baseline) to 4.29% when combining both SVAD and OSD losses.
  • Real Data (CALLHOME): The combined model achieved a DER of 8.67%, down from the baseline's 10.46%. This also outperformed the more complex RX-EEND state-of-the-art model, which scored 9.04% on their implementation.

Additionally, the ablation study proved that selecting heads based on their similarity to the identity matrix (using the trace heuristic) yielded superior results compared to randomly selecting heads for auxiliary losses.

Limitations and Future Work

While highly effective, the paper focuses exclusively on two-speaker diarization tasks. Extending this framework to an unknown or variable number of speakers remains an open challenge, as generating target masks for SVAD and OSD scales in complexity with more speakers. Furthermore, the hyperparameters balancing the loss functions ($\alpha$ and $\beta$) were fixed at 1; optimizing these dynamically could yield further performance improvements.

Summary updated

Abstract

Transformer-based end-to-end neural speaker diarization (EEND) models utilize the multi-head self-attention (SA) mechanism to enable accurate speaker label prediction in overlapped speech regions. In this study, to enhance the training effectiveness of SA-EEND models, we propose the use of auxiliary losses for the SA heads of the transformer layers. Specifically, we assume that the attention weight matrices of an SA layer are redundant if their patterns are similar to those of the identity matrix. We then explicitly constrain such matrices to exhibit specific speaker activity patterns relevant to voice activity detection or overlapped speech detection tasks. Consequently, we expect the proposed auxiliary losses to guide the transformer layers to exhibit more diverse patterns in the attention weights, thereby reducing the assumed redundancies in the SA heads. The effectiveness of the proposed method is demonstrated using the simulated and CALLHOME datasets for two-speaker diarization tasks, reducing the diarization error rate of the conventional SA-EEND model by 32.58% and 17.11%, respectively.
BibTeX
@inproceedings{jeoung2023improving,
  author = {Ye-Rin Jeoung and Joon-Young Yang and Jeong-Hwan Choi and Joon-Hyuk Chang},
  title = {{Improving Transformer-based End-to-End Speaker Diarization by Assigning Auxiliary Losses to Attention Heads}},
  booktitle = {IEEE International Conference on Acoustics, Speech and Signal Processing},
  year = {2023},
  doi = {10.1109/icassp49357.2023.10095589},
}
Figure