Neural sound localizers that work on any microphone layout spend the same computation on every part of the input. This holds whether the array is four microphones on a table, a 32-capsule sphere, or a robot head that blocks sound from reaching half its sensors. Most of the variation that matters, though, lies between microphone pairs within a single array, not between arrays. A short pair of microphones carries almost no direction information at low frequencies. A long pair produces wrapped phase that points in several directions at once at high frequencies. A pair with one microphone behind the head carries a distorted phase. We propose a localizer that splits the array into microphone-pair-and-frequency tokens and uses a mixture of experts to send each token to a specialized subnetwork, or to skip it entirely. A mixture of experts is a layer with many small subnetworks, where a learned selector activates only a few of them per input.
The core claim is narrow and testable. Learned per-token routing, keyed on each pair's physical regime and its measured coherence, should beat a dense geometry-agnostic model at matched computation on unseen arrays. It should also make compute grow with the number of informative pairs rather than with all $M(M-1)/2$ pairs. If a dense model that receives the same physical features matches it, the claim is wrong, and the experiments are built to show that.
Where this sits
Geometry-agnostic acoustic models already exist, and all of them are dense. PhaseCoder embeds each microphone's 3D coordinates into a transformer and reports 7.44° mean absolute error on the LOCATA 8-microphone robot head. Its authors note that it treats sensors as free-floating points and degrades under body shadowing. VarArray handles any number of microphones by averaging per-channel features (Transform-Average-Concatenate pooling), which makes it invariant to microphone ordering but discards absolute coordinates. (SP)2-Net feeds the array's steering vector for a hypothesized angle into a dense network to produce a continuous spatial spectrum, and is tested only on a 16-element uniform line. These papers supply our input design and our strongest baseline. None of them asks whether a single shared set of weights is the right way to handle heterogeneous pairs.
The one paper joining mixture of experts with direction-of-arrival estimation, the underwater sparsely-gated MoE estimator, motivates the work but differs from it in three ways. It routes on noise color. It uses a fixed 10-element uniform linear array. Its experts are pre-trained on hand-separated noise datasets and then frozen. It reports 0.94° RMSE at 0 dB against 6.21° for a monolithic CNN, but that comparison mixes routing with noise-label supervision and is not parameter-matched, so we do not treat it as evidence that routing itself helps. Our proposal differs on every axis: it routes on geometry-dependent pair regimes, it trains end to end, and it targets arrays never seen in training.
The broader mixture-of-experts literature constrains the method. GShard supplies top-$k$ routing with load-balancing losses. V-MoE shows that experts specialize on token types, which suggests pair tokens may do the same. Compute-Optimal Is Not Cluster-Optimal warns that saved FLOPs need not become saved wall-clock time, so we measure both.
Core Claim
The obvious combination at this intersection would route each whole array to a geometry expert. That idea is weak, because a deployed array's geometry is known and fixed. Per-array routing is then just a lookup table, and a dense model given a geometry embedding can do the same job.
The heterogeneity that a lookup cannot capture appears at the pair level. For a pair with baseline vector $\mathbf{b}_{ij} = \mathbf{r}_j - \mathbf{r}_i$, where $\mathbf{r}_i$ is microphone $i$'s position, define the baseline length in wavelengths:
$$\kappa_{ij}(f) = \frac{\|\mathbf{b}_{ij}\| f}{c}$$
Here $f$ is frequency and $c$ is the speed of sound. Phase difference is unambiguous only when $\kappa < 1/2$; above that it wraps. When $\kappa$ is near zero, the pair is almost uninformative. This regime is a deterministic function of geometry, but every array contains all three regimes spread across its pairs and frequencies.
On top of this, measured inter-channel coherence varies from one recording to the next with reverberation, signal-to-noise ratio, and shadowing. Our hypothesis is that routing on the joint signal of $\kappa$, baseline orientation, and coherence will do three things. It will let experts specialize on unambiguous, aliased, and corrupted phase. It will transfer to unseen arrays, because any new array is just a new mixture of familiar pair regimes. And it will let a zero-compute "skip" expert discard uninformative tokens, which matters because pair counts grow quadratically: 496 pairs for the 32-microphone Eigenmike against 6 for a 4-microphone array.
Research Program
Model. Each token is one microphone pair at one frequency bin. Its features are:
- the phase-normalized cross-spectrum (as in generalized cross-correlation with phase transform);
- its magnitude coherence;
- the scaled baseline vector $\mathbf{b}_{ij} f / c$.
This tokenization does not depend on microphone ordering, yet it keeps the array frame. That partly resolves the tension between PhaseCoder's absolute coordinates and VarArray's order-invariant pooling. A shallow dense encoder feeds several mixture-of-experts layers with top-1 or top-2 routing and one null expert. Direction queries then attend over the surviving tokens to produce a spatial spectrum, in the spirit of (SP)2-Net's hypothesis-conditioned scoring.
Training. We train on a synthetic room-impulse-response corpus built like PhaseCoder's: random 3D layouts of 3–8 microphones and varied room materials. We add rigid-sphere scattering on a random subset of layouts so that shadowed pairs appear in training. A compute penalty on non-null routing sets the operating point. A balancing loss in the style of GShard prevents expert collapse.
Analysis. We will measure the mutual information between expert assignment and three labels: the $\kappa$ bin, the coherence quantile, and a shadowed-or-clear label. This tests whether specialization follows physics or something incidental.
Baselines
The naive reference points are classical methods run on each test array with its known layout: steered-response-power with phase transform, and MUSIC over a dense direction grid. These need no training and set the floor. On static single-source LOCATA recordings, tuned classical submissions reach about 1.0° azimuth error (LOCATA), so on easy tasks the neural methods must justify themselves on moving and shadowed cases. We also include FRIDA (FRIDA), an analytical arbitrary-layout method, run on Pyramic and LOCATA. It is the strongest training-free competitor for arbitrary layouts, though its own paper notes it fails on coherent sources.
The strongest existing learned method is a PhaseCoder-style dense coordinate transformer. Its paper reports 7.44° mean absolute error on the LOCATA robot head, ahead of GI-DOAEnet at 7.82°. We cite that number, but for fairness we retrain a dense PhaseCoder-style encoder on our exact corpus with matched parameters and a matched tuning budget. Beating it means lower mean absolute error on held-out real arrays at equal or lower measured FLOPs and latency. As a sanity check, our model should also beat the reported 7.44° on the robot head.
Four ablations isolate the key ingredient.
- Dense pair model with the same features. It uses the same tokens, including $\kappa$ and coherence as inputs, with no routing. It is run twice, once FLOP-matched and once parameter-matched. This is the decisive control.
- Hard-coded physics router. Tokens are assigned to experts by fixed $\kappa$ and coherence bins, with no learned selection.
- Random hash routing. This is the standard control showing whether learned assignment matters at all.
- Per-array-family specialists. Separate dense models are trained per family and chosen by lookup. This serves as an oracle for whole-array specialization.
Why not just…?
Why not just give a dense model the baseline-in-wavelengths and coherence features? This is the real threat. A network can, in principle, learn to treat aliased and clean tokens differently from those inputs alone. The parameter-matched and FLOP-matched dense pair model tests exactly this. If it matches routing on accuracy, the only remaining contribution is token skipping. We would then compare that skipping against a simple coherence threshold and report honestly whether the learned version wins.
Why not just route whole arrays, or train one specialist per array type? Geometry is known at deployment, so per-array routing collapses into a lookup. Lookup also cannot help on an unseen layout, or on a robot head whose shadowed pairs change with source direction. The per-family specialist oracle, evaluated zero-shot on Pyramic and the DICIT array, measures how much is lost when the test array matches no family.
Why not just prune low-coherence pairs with a threshold and keep a dense model? This is cheap and may capture most of the compute savings. It is a baseline in its own right. Learned routing must beat it on the accuracy-versus-FLOPs curve, especially on reverberant LOCATA tasks with moving sources. There, coherence alone conflates reverberation with genuinely uninformative geometry.
Why not just use classical or analytical methods such as SRP-PHAT or FRIDA? They generalize across geometry by construction. However, classical MUSIC degraded to 14.91° RMSE at 0 dB in the underwater study (sparsely-gated MoE estimator). FRIDA cannot handle coherent sources (FRIDA). LOCATA documents classical methods breaking down on moving talkers. We run all of them on the same recordings, so the comparison stays empirical rather than assumed.
Proposed Experiments
Evaluation sets. After training only on synthetic 3–8 microphone layouts, we evaluate zero-shot on four real datasets:
- LOCATA, with its robot head, 32-microphone Eigenmike, 15-microphone DICIT planar array, and hearing-aid configurations;
- RSL2019, a 4-microphone set;
- the EPFL Pyramic 24-microphone recordings used by FRIDA.
The Eigenmike, DICIT and Pyramic sets test extrapolation beyond the training microphone counts. The robot head and hearing aids test real shadowing.
Metrics. We report azimuth mean absolute error and accuracy within 5° and 10°. For cost, we report FLOPs per second of audio and single-device wall-clock latency.
Decisive checks. 1. Routing versus dense. Learned routing must beat the parameter-matched and FLOP-matched dense pair model on held-out real arrays. This must hold most clearly on shadowed arrays, where the dense model has no input that tells it which pairs are corrupted. 2. Compute scaling. With the null expert, active compute should grow well below the quadratic pair count from 4 to 32 microphones, with no accuracy loss compared with running every token. 3. Physical specialization. Expert assignment should carry substantially more information about the $\kappa$ regime and the shadowing label than hash routing does.
If the first check fails while the second succeeds, the honest conclusion is that conditional computation buys efficiency, not accuracy, for geometry-agnostic localization, and the note would be revised to say so.
References
- PhaseCoder: Microphone Geometry-Agnostic Spatial Audio Understanding for Multimodal LLMs
- VarArray: Array-Geometry-Agnostic Continuous Speech Separation
- (SP)-Net: A Neural Spatial Spectrum Method for DOA Estimation
- Underwater acoustic vector DOA estimation in hybrid noise environments based on sparsely-gated mixture-of-experts mechanism
- GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding
- Scaling Vision with Sparse Mixture of Experts
- Compute-Optimal Is Not Cluster-Optimal: Systems-Aware Scaling for Sparse Mixture-of-Experts
- The LOCATA Challenge: Acoustic Source Localization and Tracking
- FRIDA: FRI-Based DOA Estimation for Arbitrary Array Layouts