Speaker diarization means labelling who spoke when in a recording. In real meetings and phone calls people talk over each other, and modern end-to-end systems handle this by giving every speaker a learned prototype vector, called an attractor. A frame of audio is judged to contain speaker $s$ when its embedding points close enough to that speaker's attractor. Speaker verification, the related task of checking whether two clips come from the same person, became much more accurate once embeddings were trained with angular margin losses. These losses place embeddings on a unit sphere and demand a fixed safety gap in angle between a sample and every wrong identity. Diarization has tried to borrow this idea, but so far it has worked only partly, and nobody has explained why.
This note proposes an explanation and a fix. On a unit sphere, one frame cannot sit close to two nearly perpendicular speaker prototypes at once. If two people talk simultaneously, the best the frame can achieve is a cosine similarity of about 0.71 to each; with three speakers it is about 0.58. A margin tuned for single-speaker frames therefore cannot be satisfied on overlapped frames, which are exactly where missed speech remains the dominant error. We propose to split each frame embedding into two parts. Its length decides whether anyone is speaking. Its direction decides who is speaking, using a speaker-membership margin whose threshold is set so that it can actually be met when up to $K$ speakers overlap. The loss also corrects for the imbalance between overlapped and single-speaker frames.
Core Claim
The clearest evidence for the problem is a puzzle in prior work. Robust End-to-End Speaker Diarization with Conformer and Additive Margin Penalty added an additive margin to an attractor-based diarizer and reduced DIHARD III error from 21.74% to 19.99%. However, its best system deliberately left frame embeddings unnormalized, against standard verification practice. Our reading is that an unnormalized dot product lets the embedding's length make up for the geometric shortfall at overlaps: a longer vector can clear the threshold for two attractors at once.
That workaround mixes up two different things. Embedding length ends up encoding both loudness and the number of speakers present. Any margin applied this way therefore constrains identity unevenly across frames.
The geometric limit can be stated exactly. Suppose the attractors $a_1,\dots,a_S$ are orthonormal, $x_t$ is the frame embedding, and $c_{ts}=a_s^\top x_t/\|x_t\|$ is the cosine between frame $t$ and speaker $s$. Then $\sum_s c_{ts}^2\le 1$. So if $k$ speakers are active,
$$\min_{s\ \text{active}} c_{ts} \le 1/\sqrt{k}.$$
It follows that any cosine threshold above $1/\sqrt{k}$ makes every $k$-speaker frame impossible to label correctly.
Our model therefore predicts, for frame $t$ and speaker $s$,
$$p_{ts}=\sigma(w\log\|x_t\|+b)\cdot\sigma\big(\gamma(c_{ts}-\tau)\big),\qquad \tau=\kappa/\sqrt{K},\ \kappa<1.$$
The symbols are defined as follows:
- $\sigma$ is the logistic function, and $w,b$ are a learned scale and offset.
- $\gamma$ is the usual scale used in angular losses.
- $K$ is the maximum number of simultaneous speakers the system is designed for.
- $\tau$ is set below the feasibility bound.
The length term is trained with binary cross-entropy against a speech/non-speech label. The direction term is trained only on speech frames, after the speaker-to-output matching chosen by permutation-invariant training (the loss is computed under whichever assignment of model outputs to true speakers gives the lowest error). The direction term uses a binary angular margin $m$: active speakers must reach $c_{ts}\ge\tau+m$ and inactive ones must stay below $\tau-m$.
A second problem remains. Positive cosines naturally cluster near 1 for single-speaker frames and near $1/\sqrt{k}$ for overlapped ones, so the spread of positive and negative angles differs. Semi-supervised Multi-label Learning with Balanced Binary Angular Margin Loss showed that this kind of variance mismatch makes decision boundaries drift, using image and text data. We reuse its Gaussian re-balancing, but apply it separately within each overlap-degree group. That paper supplies the method. Our contribution is the overlap-degree conditioning and the separation of length from direction, neither of which it addresses.
Research Program
The base system is held fixed so that only the loss geometry changes. Frames come from a weighted sum of layers of a frozen WavLM. WavLM was pre-trained to predict the clean primary speaker under simulated overlap, and its layer analysis places speaker information in the lower and middle layers. Code is available at microsoft/unilm.
A Conformer encoder sits on top of these features; a Conformer is a Transformer block with an added convolution for local acoustic detail. End-to-End Neural Diarization: From Transformer to Conformer showed this encoder lowers diarization error on simulated data, but it also found Conformers are sensitive to the mismatch between simulated and real conversations. Mixed real and simulated training fixed this and reached 6.82% DER (diarization error rate) on 2-speaker CALLHOME.
Attractors are produced by an encoder-decoder module, and we add a soft orthogonality penalty on active attractors as in End-to-End Diarization utilizing Attractor Deep Clustering. That paper is the closest prior work. Its loss aligns frames with attractors, keeps active attractors perpendicular, and pushes inactive ones toward zero length. It reaches 4.99% DER on CALLHOME, of which 2.45% is missed speech.
The difference from that work is deliberate. Attractor Deep Clustering constrains the attractors, while we constrain how a frame relates to a set of attractors. Our margin is set so it can be met at every overlap degree up to $K$. We also give activity detection a separate magnitude channel instead of reusing attractor length to signal absent speakers.
The work has three parts. The first is analytical. We measure, on trained versions of existing loss variants, the distribution of $c_{ts}$ for active speakers broken down by overlap degree. This tests whether missed speech on $k$-speaker frames tracks the share of positives falling below each system's effective threshold. The second part is the loss itself, including choosing $\kappa$ and testing whether a learned per-degree threshold is ever needed. The third part asks whether separating length from direction makes simulation-to-real overlap mismatch easier to correct. We test this by recalibrating only $w,b$ on a small amount of real data.
The original end-to-end diarization paper, End-to-End Neural Speaker Diarization with Permutation-Free Objectives, attributed its false alarms to exactly this mismatch, so the third part is worth testing.
Baselines
Do-nothing reference. The do-nothing reference is the identical WavLM-plus-Conformer attractor system trained with standard permutation-invariant binary cross-entropy on dot products. It shares the data, schedule, and tuning budget of every other system, so any improvement over it comes from the loss alone.
Additive margin on unnormalized embeddings. This is the recipe of Robust End-to-End Speaker Diarization with Conformer and Additive Margin Penalty, reimplemented on our backbone. It is also run with both frames and attractors normalized, to reproduce the reported degradation. It is the most direct competitor because it already shows margins help diarization and implicitly uses length to cope with overlap. On DIHARD III Track 2 full evaluation it reports 19.99% DER, against 21.74% for its EEND-EDA baseline.
Attractor Deep Clustering. We run the orthogonality and zero-suppression loss of End-to-End Diarization utilizing Attractor Deep Clustering on the same backbone. It is the strongest reported CALLHOME system in the file, at 4.99% DER with 0.27% speaker confusion and 2.45% missed speech. Its original setup used a CNN front end rather than WavLM, so we also compare against the published number directly.
Power-set encoding. Following SEND, we treat each combination of up to $K$ speakers as one class and train it with an ArcFace-style margin over power-set prototypes. SEND reports 4.88% DER on AliMeeting.
Ablations. Two ablations isolate our ingredients. The first keeps the overlap-consistent threshold but drops the separate length channel. The second keeps the length channel but uses a fixed threshold near 1 without per-degree re-balancing.
Hyperparameter tuning. All systems receive the same budget, roughly twenty configurations over margin, scale, and learning rate, selected on development sets only. Results average three seeds.
What counts as success. Beating the strongest baseline means a lower overall DER on CALLHOME and DIHARD III, with the reduction concentrated in missed speech on frames with two or more speakers. A seed-consistent drop in 3- and 4-speaker missed speech, with no loss on single-speaker frames, would count even if overall DER moves only modestly. No reported result found on Lacuna for a WavLM-front-end end-to-end diarizer on CALLHOME, so our do-nothing reference establishes that point.
Why not just…?
Why not just keep unnormalized dot products, since that already works? This is the main threat. Length can technically absorb the overlap shortfall, which is presumably why Robust End-to-End Speaker Diarization with Conformer and Additive Margin Penalty preferred it. Our hypothesis is that it does so inconsistently: quiet overlapped speakers produce short vectors and get missed. The per-degree breakdown of missed speech, together with a check for correlation between embedding length and secondary-speaker energy on simulated mixtures, decides the question. If the unnormalized baseline matches us on 3- and 4-speaker frames, the geometric argument does not matter in practice and we will report that.
Why not just use power-set classes, which make overlap single-label so ArcFace applies directly? Power-set encoding avoids the infeasibility problem. However, each combination becomes its own unrelated prototype, so nothing ties the "A and B together" class to the solo classes "A" and "B". The number of classes also grows combinatorially with $K$. SEND relies on pre-extracted speaker embeddings, as noted in SEND. We run this as a full baseline. If it wins at $K\le 3$ on AliMeeting, the right conclusion is that our approach matters mainly as $K$ grows, which we can test by raising $K$ to 4.
Why not just tune detection thresholds per overlap degree after training? Post-hoc thresholds cannot repair an embedding space trained under a margin that could never be met. The overlapped positives were pushed toward a target they could not reach, so their angles end up scattered rather than tightly clustered. We test this directly by giving the normalized-margin baseline oracle per-degree thresholds and comparing.
Why not just separate the speakers first, then verify each stream? Separation adds an entire extra network. Enroll-Aware Attentive Statistics Pooling for Target Speaker Verification argues this front end is redundant and slow. It is also outside the scope of a question about loss geometry, so we do not run it.
Proposed Experiments
Training data. Training uses simulated mixtures with up to four speakers, followed by adaptation on mixed real and simulated data in the manner of From Transformer to Conformer.
Evaluation sets. We evaluate on four sets:
- CALLHOME, which has low natural overlap.
- DIHARD III Track 2, with diverse domains, as described in The Third DIHARD Diarization Challenge, where multi-party domains show median DERs of 35–45%.
- AliMeeting, with 34–42% overlap.
- Simulated sets with controlled overlap degree.
Metrics. We report DER split into missed speech, false alarm, and confusion, and additionally stratified by the number of active speakers per frame.
Decisive checks. The first check is the feasibility prediction. Under a normalized fixed-margin loss, missed speech should rise sharply at 3 and 4 speakers. Our loss should flatten that curve, and the active-speaker cosine histograms should centre above $\tau$ at every overlap degree. The second check is the per-degree breakdown of the unnormalized baseline described above. The third check uses the WavLM front end itself. Because WavLM was pre-trained against overlap, one might expect it to make loss geometry irrelevant, so we repeat the main comparison with a filterbank front end. If the gain disappears only with WavLM, the result says something about representations rather than losses.
The fourth check tests whether separating length from direction helps with simulation-to-real transfer. We recalibrate only the length channel on CALLHOME adaptation data and compare against full fine-tuning. Success would mean recovering most of the false-alarm reduction while touching only two parameters.
References
- Robust End-to-End Speaker Diarization with Conformer and Additive Margin Penalty
- Semi-supervised Multi-label Learning with Balanced Binary Angular Margin Loss
- WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing
- End-to-End Neural Diarization: From Transformer to Conformer
- End-to-End Diarization utilizing Attractor Deep Clustering
- End-to-End Neural Speaker Diarization with Permutation-Free Objectives
- Speaker Embedding-aware Neural Diarization: an Efficient Framework for Overlapping Speech Diarization in Meeting Scenarios
- Enroll-Aware Attentive Statistics Pooling for Target Speaker Verification
- The Third DIHARD Diarization Challenge