Look-Ahead Budgets for Moving Sound Source Localization

Neural systems that locate moving sound sources work out where a talker or object is from multichannel microphone recordings. They increasingly owe their gains to sequence models that read long stretches of audio at once, such as attention over seconds of frames, bidirectional recurrence, and state-space memory. The forecasting, clinical and recommender literatures call it temporal leakage when a model is scored as if it could see the future but deployed where it cannot, and they found that this inflates results. We argue that the same thing is happening in a subtler form inside localization pipelines. A tracker that sees where the source will be 500 ms from now can interpolate its current position. A centered spectrogram window, a normalization computed over the whole clip, a bidirectional layer, or a post-hoc smoothing step all quietly grant that preview. Current papers report a single error number with no statement of how much future audio each estimate used.

We propose to make that hidden quantity explicit. Every component of a localization pipeline is restricted to a look-ahead budget $L$, meaning the maximum milliseconds of audio after time $t$ that may influence the direction estimate at $t$. Error is then reported as a curve over $L$ rather than as one number. The central claim is testable. We expect that a substantial share of the reported advantage of sequence-model localizers on moving sources is a look-ahead dividend: it disappears at deployable latencies and is largely reproduced by simply smoothing frame-wise estimates offline. Under a matched budget, the ranking between heavy architectures and simple causal trackers may change. The deliverable is an evaluation protocol plus a training recipe that lets one model be evaluated honestly at any budget.

Where the two directions actually meet

The moving-source localization literature already knows that frame-independent models fail on moving sources. The file's summaries of Baek et al. and Argentieri et al. describe frame-independent models without recurrent memory degrading sharply when sources move. Jiang et al. points to a window-length trade-off: short windows lack spectral resolution, while long windows let the source move during the window. The response has been longer temporal context. DeFT-Mamba uses state-space sequence layers, which carry a running memory across time at linear cost, to separate and classify sources moving up to 3 m/s. Triple-path attentive recurrent networks such as Pandey et al. model time, frequency and channel jointly. These papers motivate the proposal: they show that temporal context helps. In the file's summaries, however, they do not say how much of that context lies in the future relative to the frame being scored.

The temporal-leakage literature supplies the lens and the main warning signs. Set Functions for Time Series found that global normalization and bidirectional imputation in clinical baselines leak future values and fail in online monitoring. Its fix is masked, causal processing, and that is exactly the kind of component-level audit we transplant to spectrogram pipelines. Aeolus measured an average AUC inflation of 0.057 from random versus chronological splits. Evaluating Model Performance in Medical Datasets Over Time and Beyond Leakage and Complexity report that complex models lose their edge, sometimes falling behind simple ones, once leakage is removed. These results motivate our ranking-inversion hypothesis. They also constrain the design, because all of them concern train/test split leakage over months or years, which is not the main mechanism at the millisecond scale of acoustics.

The research file found no paper joining the two areas. The closest acoustic work is The LOCATA Challenge. It benchmarks localization and tracking of static and moving sources with motion-capture ground truth, and compares instantaneous estimators against Kalman, particle and related tracking filters. LOCATA establishes the classical filtering-versus-estimation comparison. It does not, as summarized here, index modern neural systems by how much future audio they consume. Our work differs from LOCATA by treating look-ahead as a controlled axis applied uniformly to neural and classical pipelines. It differs from the leakage literature by locating the leak in inference-time context rather than in data splitting.

Core Claim

Let $E(L)$ be the localization error of a pipeline when no component may use audio later than $t+L$ to estimate the direction at $t$. We define the look-ahead dividend as

$$D = E(0) - E(L_{\text{full}}),$$

where $L_{\text{full}}$ is the unrestricted context the published model actually uses. We claim three things. First, $D$ is large for current sequence-model localizers on moving sources and grows with source speed, because interpolating between past and future positions is most valuable when the source moves fast. Second, most of $D$ can be recovered by a frame-independent localizer followed by a fixed offline smoother given the same $L$, so it reflects preview rather than learned motion modeling. Third, at small $L$, simple causal trackers close much of the gap to heavy architectures. Any of the three can fail, and each failure is informative. A small $D$ would vindicate current numbers. A large $D$ that the smoother cannot recover would be positive evidence that sequence models learn genuine motion structure.

Research Program

The first component is a leakage audit and budget-enforcement library for multichannel localization pipelines. We enumerate each place where future audio can enter and give each a causal or bounded-look-ahead variant. These places are: STFT framing (centered versus trailing windows), feature normalization (whole-clip statistics versus running statistics), learned layers (bidirectional recurrence, unmasked attention, bidirectional state-space scans), and post-processing (label smoothing, track smoothing, voice-activity gating computed over the clip). Budget $L$ is enforced by masking attention beyond $t+L$, using a forward scan with a fixed $L$-frame delay for recurrent and state-space layers, and limiting STFT and normalization to the same horizon. We also instrument a gradient probe that measures how much the output at $t$ depends on input after $t+L$. This verifies that enforcement is real rather than nominal.

The second component is look-ahead-conditioned training. Training a separate model per budget repeats the compute problem that EMDOT notes for rolling backtests. Instead, during training we sample $L$ per batch from a range spanning 0 to $L_{\text{full}}$, apply the matching mask, and feed $L$ to the network as a conditioning input. A single trained model then yields the whole curve $E(L)$. Whether this matches dedicated per-budget models is an explicit ablation, not an assumption.

The third component is analysis. We compute $E(L)$ for each architecture, binned by source angular velocity and by reverberation, and compare it against the smoother control described below. We also separate preview gains from motion-memory gains: a causal model with long past context but $L=0$ isolates the value of memory, and a model with short past context but large $L$ isolates the value of preview.

Baselines

The naive reference point is a frame-independent convolutional-recurrent localizer run causally (the "do nothing about time" model), since the curve's lower anchor needs it. Deeply Supervised Curriculum Learning reports such a standard CRNN on LOCATA at 86.98% accuracy and 8.83° MAE. We reimplement it and check that we reproduce that regime before trusting any curve.

The strongest existing localizer reported in the file on LOCATA is DSCL itself, at 94.88% accuracy and 2.94° MAE in that same paper. For cross-array generalization the strongest is AGG-RL, at 90.78% within-10° accuracy on the NAO head versus 86.25% for the best baseline, and 11.24° MAE zero-shot on the Eigenmike. AGG-RL is the most relevant competitor because it also reports degradation on moving sources in STARSS23. DSCL is the strongest on the exact benchmark we use, so it is the primary target. A long-context sequence localizer built from DeFT-Mamba-style gated convolution plus state-space blocks represents the "temporal memory" family. DeFT-Mamba reports separation and classification (7.27 dB SDR, F1 60.3), not localization error, and I found no reported localization result on Lacuna for it. We therefore attach a direction-of-arrival head and report it as our adaptation, not as a published number. All neural models are trained on identical data with the same number of GPU-hours and the same hyperparameter search budget. Each is evaluated at its published context and at every enforced $L$. Beating DSCL means lower MAE at a deployable budget (we fix $L \le 100$ ms in advance as the deployment point) on LOCATA moving-source tasks. No published moving-source number exists in the file for DSCL on these tasks, so the comparison is against our own reproduction.

The classical reference is steered-response-power or MUSIC direction estimation followed by a Kalman filter, as compared in LOCATA. It is causal by construction. LOCATA reports roughly 1.0° azimuth error for top methods on the static single-source task, but I found no reported result on Lacuna for classical trackers on the moving-source tasks.

The key ablation is the offline-smoother control: the frame-independent localizer followed by a fixed-lag smoother, the backward-pass counterpart of the Kalman filter, allowed exactly $L$ of future frames. If a sequence model's advantage at budget $L$ does not exceed this control's advantage at the same $L$, its gain is attributable to preview. A second ablation compares look-ahead-conditioned training against separately trained per-budget models at three budgets.

Why not just…?

Why not just train and report a causal model? That is one point on our curve, and we do it. A single causal number cannot tell readers whether the published offline number reflected motion modeling or preview, and it cannot show where the architecture ranking flips. The smoother control and the velocity-binned $D$ supply that. If separately trained causal models clearly outperform the conditioned model at $L=0$, we will say so and recommend per-budget training. The protocol does not depend on the conditioning trick.

Why not just split by room or recording, as audio researchers already do? This is a real threat to one part of the leakage story, and we concede it in the design. Splitting by recording likely removes most train/test leakage of room responses and trajectory ranges. That is why our main claim targets inference-time look-ahead, which no split can remove. We still run a control on STARSS23 and LOCATA comparing random-segment splits against recording-level splits. If the gap is negligible, we report that split leakage is a non-issue in these benchmarks and keep the focus on look-ahead.

Why not just use classical localization with a Kalman filter, which is causal anyway? Classical pipelines are leak-free, but the file's evidence shows they are much less accurate. Neural-SRP reached 52.75° MAE versus AGG-RL's 11.24° on an unseen Eigenmike, and LOCATA notes that tracking moving talkers in reverberant rooms stayed hard for filtering methods. They serve as our causal floor. The open question is how much of the neural margin survives at matched budget, not whether classical methods win outright.

Why not just discard order and treat frames as a set? Set models like SeFT avoid sequential leakage in clinical data. For moving sources, though, the ordering of frames carries the trajectory, so a set model gives up the motion information we want to measure. It does not resolve the latency question.

Proposed Experiments

We evaluate on LOCATA moving-source tasks (Tasks 3 to 6, including moving arrays), using azimuth MAE and within-10° accuracy against motion-capture ground truth. We also evaluate on STARSS23, using its class-dependent localization error and recall, where the reported audio-only error in the tetrahedral format is 66.9° (STARSS23). Finally, we use a simulated-room set modeled on the DeFT-Mamba setup: a four-channel tetrahedral array, varying reverberation, and sources up to 3 m/s. There we control velocity exactly and can extend it to stress-test the speed dependence of $D$.

The first decisive check is whether $D$ exceeds the smoother control's dividend at matched $L$ for each sequence model, reported per velocity bin with bootstrap confidence intervals over recordings. A sequence model that beats the smoother by a clear margin at $L=0$ has learned motion. One that only matches it has learned to peek.

The second check is whether the ranking of DSCL, AGG-RL-style, state-space and classical-tracker pipelines at $L \le 100$ ms differs from the ranking at full context. This tests whether the complexity inversion seen in Beyond Leakage and Complexity reappears in acoustics.

The third check is the gradient probe. It must show near-zero dependence beyond $t+L$ for every enforced pipeline; otherwise the curve is not trustworthy.

The fourth check is the split-leakage control described above. Together these yield a reporting standard, error as a function of look-ahead, that any future moving-source localization paper can adopt at the cost of one extra training run.