Picture a small device with a few microphones that must pull one known person's voice out of a room full of talkers. It has two kinds of evidence. One is where the sound comes from: the tiny arrival-time differences between microphones. The other is who is talking: the voiceprint learned from a short enrollment clip. Efficient speech models already spend more compute on some frequency bands than others. They almost always give low frequencies fine resolution and high frequencies coarse resolution, on the assumption that low frequencies are where "the information" lives. That assumption is about a single kind of information. Inter-microphone timing cues are physically trustworthy only below a cutoff set by microphone spacing. Speaker identity cues need not follow the same profile. This note proposes to measure, band by band, how much spatial and identity evidence each frequency region carries, and to build a target speaker extractor that spends its compute and its conditioning according to those two separate maps. Location carries the selection where location is reliable. Per-band voiceprints carry it where location breaks down.
The core claim is testable and could fail. Under a fixed compute budget, a multi-channel extractor whose frequency allocation follows cue-specific information maps, with spatial features only below the aliasing frequency and band-specific speaker embeddings above it, should beat both uniform full-band models and the standard "fine-low, coarse-high" band split. The gain should show up specifically on test mixtures whose speakers and rooms were never seen in training. If a single global voiceprint broadcast to all bands does as well, the idea is wrong, and the experiments are designed to show that.
What exists and what is missing
Three lines of work each cover two of the four ingredients. They share a blind spot.
Personalized speech enhancement combining band-split RNN and speaker attentive module splits the spectrum into non-uniform bands and uses one pretrained ECAPA-TDNN vector to reweight every band. It is single-channel, and it applies the same full-band voiceprint to bands that may carry very different amounts of identity. That paper supplies our backbone. Our change is per-band conditioning and spatial input.
A Lightweight and Real-Time Binaural Speech Enhancement Model with Spatial Cues Preservation (LBCCN) keeps only the lowest 40 frequency bins, where binaural timing cues live. It reaches competitive enhancement with 38k parameters. It has no notion of a target speaker, and its hard truncation throws away whatever identity evidence sits higher up. LBCCN motivates the low-band spatial path and constrains the design: we do not discard high bands, we change what they are used for.
MC-LExt and Direction-Aware Speaker Beam do multi-channel target extraction with full-band uniform processing. MC-LExt prepends raw enrollment audio to every channel, which makes it accurate but computationally heavy. Direction-Aware Speaker Beam relies on 12 fixed beams for one array geometry.
On the identity side, Hierarchical Speaker Representation for Target Speaker Extraction shows that adding local frequency-axis speaker features to a global ECAPA vector helps, by 0.493 dB SI-SNR over TEA-PSE on Libri-2talker. That is evidence that one global vector is a bottleneck. Its local features are not organized by band, are not tied to compute allocation, and are single-channel.
The efficiency case for band splitting is already strong. TIGER matches or exceeds TF-GridNet on EchoSet (14.22 vs 13.73 dB SDRi) with 0.82M parameters and 15.27 G/s MACs, against 14.43M and 323.75 G/s. TIGER's split is a fixed, finer-at-low-frequencies design chosen for blind separation. Nobody has asked whether the right split changes when two different cues compete for each band.
Core Claim
Spatial evidence has a hard physical ceiling. For two microphones spaced $d$ apart and sound speed $c$, the inter-channel phase difference identifies direction without ambiguity only below
$$ f_{\text{alias}} = \frac{c}{2d}. $$
This follows because the phase difference wraps once the path difference exceeds half a wavelength. For a 5 cm spacing $f_{\text{alias}}$ is about 3.4 kHz. For a head-sized binaural spacing it is near 1 kHz.
Above $f_{\text{alias}}$, a model that wants to keep tracking the target must rely on something other than direction. We hypothesize that speaker identity is that something, and that band-specific voiceprints carry it better than a single full-band vector. The proposal therefore builds the network around a measured crossover. Low bands are fine-grained and spatially driven. Bands above the crossover are conditioned on per-band speaker embeddings, with a cross-band module that propagates the low-band target selection upward. Band widths are set by the measured maps rather than by mel or Bark spacing.
Research Program
Measurement. The first stage measures the two maps.
For identity, we train band-limited speaker encoders on VoxCeleb. These are ECAPA-TDNN-style models that each see only one frequency band, as in ECAPA-TDNN. We evaluate them on VoxCeleb1 test speakers disjoint from training and obtain an equal error rate per band. The equal error rate is the operating point where false accepts equal false rejects. For a reference point, the full-band ECAPA-TDNN reports 0.87% on VoxCeleb1. We also measure how much each band adds to a full-band encoder, so that redundancy between bands is visible.
For space, we simulate reverberant rooms in the style of WHAMR! for several array spacings. We train small per-band classifiers that use only inter-channel phase and level differences to decide whether a time-frequency cell is dominated by the target or the interferer. We score them on held-out rooms.
The deliverable is a pair of curves per array geometry, and the crossover band where identity evidence overtakes spatial evidence. All measurements use speaker-disjoint and room-disjoint folds, repeated over several random speaker partitions. The speaker-independent literature warns that results on modest speaker pools vary with which speakers are held out, so we report the spread.
Model. The second stage builds the extractor.
The backbone is a band-split recurrent network following the personalized band-split RNN paper. Below the crossover, each band receives the multi-channel complex spectrum plus phase-difference features at fine resolution, in the spirit of LBCCN. Above the crossover, bands are coarser and drop spatial features. Each such band is instead queried by a lightweight per-band speaker embedding, distilled from the stage-one band encoders and computed from a 3-second enrollment clip. A small cross-band attention layer lets the target mask estimated at low frequencies inform the high bands.
We train the allocation in two variants. One fixes it from the measured maps. The other is initialized from the maps and fine-tuned under a MAC penalty. Comparing the two tells us whether measurement alone suffices.
Evaluation. The third stage is the evaluation protocol, which addresses a confound the field has not separated. Standard speaker-disjoint splits on simulated multi-channel data also randomize rooms, so "unseen speaker" and "unseen room" are tangled together. What Do Classifiers Actually Learn? shows how badly shortcuts can inflate scores: UAR on eNTERFACE drops from 83.8% to 28.4% once a confounding cue is removed. A multi-channel extractor could similarly memorize room transfer functions instead of linking voiceprint to target.
We therefore regenerate MC-Libri2Mix-style 4-channel mixtures and a 2-channel WHAMR!-style set at 16 kHz. The 16 kHz rate keeps the high bands that identity may depend on. Mixtures are built on a factorial grid: speakers seen or unseen, crossed with rooms seen or unseen, plus a held-out array spacing. The grid lets us attribute any drop to new voices, new rooms, or new geometry. Our method predicts its advantage is largest in the unseen-speaker, unseen-room cell. If the band-specific voiceprints merely memorize training speakers, the advantage will vanish there.
Baselines
The naive reference is the unprocessed reference-channel mixture, which defines 0 dB SI-SDR improvement. Alongside it we run a mask-based MVDR beamformer driven by an oracle target mask. That gives an upper bound on what pure spatial filtering can do. Direction-Aware Speaker Beam reports that its oracle MVDR is about 1.2 dB above its own system, a sign that spatial filtering alone is a serious reference.
The strongest existing method is MC-LExt, which reports 16.3 dB SI-SDRi on 4-channel MC-Libri2Mix and 19.1 dB on WHAMR!. On WHAMR! its standard embedding-conditioned counterpart reaches 17.4 dB and the monaural baseline 17.1 dB, all from MC-LExt. It is the strongest competitor because it is the best reported multi-channel target extractor in the file, and because it uses raw enrollment audio rather than a lossy vector. We first reproduce its published numbers on the original benchmarks to validate our pipeline. We then retrain it on our factorial splits with the same data, the same number of training steps, and the same hyperparameter-search budget as our model, and report its MACs.
For blind multi-channel separation on spatialized WSJ0-2mix, UniArray reports 13.76 dB SI-SDRi on the seen 8-channel array and 13.46 dB on unseen geometries. It is not a target extractor, so it serves only as a geometry-robustness reference. For the TIGER-style allocation with phase features added, there is no reported multi-channel target-extraction result on Lacuna, so we implement and tune it ourselves at matched MACs.
Two ablations isolate the key ingredient. In the first, the same backbone and band layout get spatial features in all bands and a single global ECAPA vector broadcast to every band, as in the personalized band-split RNN. In the second, everything else is held fixed but per-band embeddings replace the global vector. A third control keeps per-band embeddings but uses mel-spaced, fine-low bands instead of the measured allocation.
Beating the field means one of two outcomes. The first is coming within 0.5 dB SI-SDRi of the retrained MC-LExt at a tenth or less of its MACs. The second is exceeding every matched-MAC baseline by at least 1 dB in the unseen-speaker, unseen-room cell, with the gap larger there than in the seen-speaker cells. Each comparison is averaged over several speaker folds, and the gap must exceed the spread across folds.
Why not just…?
Why not just use a large uniform full-band model and let it learn where to look? A big enough TF-GridNet or MC-LExt can in principle learn cue-specific weighting on its own, so at unlimited compute this proposal may add nothing to accuracy. The claim is about a fixed budget. TIGER's result shows uniform models waste most of their compute: comparable SDRi at about 5% of the MACs. We report full accuracy-versus-MACs curves, so a reviewer can see where, if anywhere, the uniform model catches up.
Why not just add phase-difference features to a band-split network with one global ECAPA vector? This is the most serious threat, because it is a short step from the personalized band-split RNN and needs no new embedding machinery. The evidence against sufficiency is indirect. HR-TSE gains 0.493 dB from local features. MC-LExt gains 1.7 dB on WHAMR! by replacing an embedding with raw enrollment audio. Neither shows that the missing information is band-specific. The first ablation in Baselines settles this directly. If per-band embeddings do not beat the broadcast global vector above the crossover, especially for same-gender pairs where global vectors are least distinctive, we drop that component and report the measurement study and allocation as the contribution.
Why not just estimate masks at low frequencies and let a beamformer handle the high bands? Beamformers driven by covariance matrices estimated from masks do not require unwrapped phase, so they may extend low-band selection upward without any identity conditioning. The oracle-mask MVDR baseline and a variant that uses our own low-band masks fed to MVDR test this. If that variant matches our model above the crossover, the voiceprint path is unnecessary and the cheaper design wins. We would say so.
Decisive Checks
Three results decide whether the core claim holds.
The measurement must first show a real crossover: per-band spatial informativeness collapsing near $f_{\text{alias}}$ while per-band speaker equal error rates stay usable above it on disjoint VoxCeleb1 speakers. If identity evidence also concentrates at low frequencies, the complementary-allocation premise fails. The study then reduces to a negative result about cue overlap, which is still informative for designs like LBCCN.
Second, on the factorial MC-Libri2Mix and WHAMR!-style grids, the per-band-embedding model must beat the global-vector ablation at matched MACs. The gain must be concentrated in high bands, checked by band-limited SI-SDR, and in the unseen-speaker, unseen-room cell.
Third, moving the crossover with array spacing should move the optimal allocation in the direction $c/2d$ predicts. That is a physical prediction a purely learned model has no reason to satisfy by accident.
References
- Personalized Speech Enhancement Combining Band-Split RNN and Speaker Attentive Module
- A Lightweight and Real-Time Binaural Speech Enhancement Model with Spatial Cues Preservation
- MC-LExt: Multi-Channel Target Speaker Extraction with Onset-Prompted Speaker Conditioning Mechanism
- Direction-Aware Speaker Beam for Multi-Channel Speaker Extraction
- Hierarchical Speaker Representation for Target Speaker Extraction
- TIGER: Time-frequency Interleaved Gain Extraction and Reconstruction for Efficient Speech Separation
- ECAPA-TDNN: Emphasized Channel Attention, Propagation and Aggregation in TDNN Based Speaker Verification
- WHAMR!: Noisy and Reverberant Single-Channel Speech Separation
- What Do Classifiers Actually Learn? a Case Study on Emotion Recognition Datasets
- UniArray: Unified Spectral-Spatial Modeling for Array-Geometry-Agnostic Speech Separation