Moving Talkers as Free Exploration: Greedy Listener-Preference Learning for Speech Enhancement

A hearing aid or conferencing device that adapts its noise suppression to a particular listener faces a hard trade-off. To learn what the listener prefers, it has to try settings and observe reactions. Deliberately playing a setting it believes is worse ("exploring") means playing audibly worse audio to a real person. This proposal asks whether the device can skip deliberate exploration entirely. The idea is to pick, at every moment, the setting it currently thinks is best, and let the world supply the variety needed to learn. The source of that variety is something acoustics usually treats as a problem: the talker moves. As a talker walks and turns, the direction, distance and reverberation of their voice keep changing. Which enhancement setting sounds best keeps changing with them, so a purely greedy learner is pushed to try many settings in many conditions without ever being told to.

The claim is that source motion, which breaks classical "stationary source" signal processing, is exactly the kind of natural context variation that makes exploration-free learning provably and practically work. The trick is to treat the talker's spatial state as observed context rather than as drift in the unknown preference model. We test this with online learners that choose among a handful of enhancement pipelines from sparse perceptual feedback. The learners use high-dimensional audio features, the reward is a perceptual rather than energy-based quality score, and the audio comes from simulated and real moving-talker recordings.

Core Claim

Two terms first. A contextual bandit is an online learner that sees some features ("context") each round, picks one action, and observes a reward only for that action. Regret is the total reward lost relative to always picking the best action. Theory already shows that a greedy linear bandit, one that fits a regression and picks the argmax with no exploration bonus, achieves low regret when contexts are diverse enough to span the feature space. It fails when they are not.

We claim three things. First, when the talker's spatial state is included in the context, a moving talker supplies enough diversity for greedy selection to succeed where a static talker traps it. Second, a sparse, thresholded greedy learner keeps this advantage when the context has on the order of a thousand dimensions, such as self-supervised speech embeddings plus spatial covariance summaries. Third, the benefit survives when the reward is a perceptual score that separates leakage from distortion, rather than signal-to-distortion ratio.

The reward model we assume is

$$r_t = \langle x_t, \theta_{a_t} \rangle + \varepsilon_t, \qquad x_t = [\,s_t,\; z_t\,],$$

where $r_t$ is the listener's perceptual reward for the audio block played at round $t$, $a_t$ is the chosen pipeline, $\theta_a$ is an unknown sparse preference vector for pipeline $a$, $\varepsilon_t$ is noise, $s_t$ is the talker's estimated spatial state (direction, direction change rate, distance proxy, direct-to-reverberant ratio), and $z_t$ are acoustic features of the current mixture.

The central design decision is that motion enters through $s_t$, while $\theta_a$ stays fixed. The alternative is to let $\theta$ drift over time and chase it with forgetting windows. We argue that treating motion as observed context is both more sample-efficient and what turns motion into free exploration.

Relation to Prior Work

The exploration-free theory supplies the method. Local Anti-Concentration Class: Logarithmic Regret for Greedy Linear Contextual Bandit proves that plain least-squares greedy achieves polylogarithmic regret under a smoothness-of-density condition on contexts, and reports it beating LinUCB and Thompson sampling in simulation. Stochastic Linear Contextual Bandits with Diverse Contexts and Mostly Exploration-Free Algorithms for Contextual Bandits show that what matters is whether the regions where each arm is chosen span the feature space. Greedy Algorithms for Structured Bandits: A Sharp Characterization of Asymptotic Success / Failure explains failure through "decoys," wrong models that agree with the truth on the arm being played, which context diversity destroys.

All of these assume fixed parameters and abstract, stationary context distributions. None asks whether a physical process such as a talker's trajectory produces the required diversity. That is the empirical question we answer.

For high dimensions, Thresholded Lasso Bandit gives a greedy learner whose regret grows additively in $\log d$. It fits a Lasso, thresholds away spurious features, and refits least squares on the surviving support, and it is validated up to $d=10{,}000$ in simulation and on Yahoo! R6A. New Classes of the Greedy-Applicable Arm Feature Distributions in the Sparse Linear Bandit Problem widens the feature distributions for which this holds. We reuse the thresholded greedy learner as our core algorithm. Its assumptions of symmetric, balanced covariance are likely violated by speech embeddings and trajectory-driven features, so we measure how much that matters rather than assume it away.

One result constrains the design. Is Pure Exploitation Sufficient in Exogenous MDPs with Linear Function Approximation? shows pure exploitation works when randomness is exogenous, meaning it evolves independently of the agent. It also proves linear regret when the reward or the agent-controlled dynamics are unknown. A moving talker is exogenous in exactly this sense. Our reward, however, is unknown, and recursive beamformers have state that would depend on which output we chose.

We avoid the negative result structurally. Every candidate pipeline keeps its own internal state and runs in parallel, and the learner only chooses which output to play. This removes agent-controlled dynamics and leaves a pure contextual bandit, where greedy theory with unknown rewards applies.

The acoustics papers motivate the setting. Beamformers that assume a fixed look direction cancel a moving target (Maximum Negentropy Beamforming). ARiSE: Auto-Regressive Multi-Channel Speech Enhancement tracks targets by feeding back beamformed estimates, but notes it degrades under fast motion. SonicSim and RealMAN supply simulated and real moving-source data and document large simulation-to-real drops. The ARiSE limitation tells us that different pipelines win at different motion speeds, which is what makes the choice among them worth learning.

The perceptual metric supplies the reward. MAPSS maps self-supervised embeddings onto a manifold and reports two scores, Perceptual Match for self-distortion and Perceptual Separation for leakage, at 75 frames per second. These rank first or second in correlation with human ratings on SEBASS. The split matters for us: listeners plausibly differ in how much they penalize leakage versus distortion, a preference that signal-to-distortion ratio cannot express. SpeechJudge and MARS-Sep are cautionary. General audio judges agree with humans below 70%, and policies optimized against neural rewards can hack them. Our learner only selects among fixed pipelines and cannot synthesize new audio, so the room for reward hacking is small by construction. We still audit it.

Research Program

We build a streaming system that runs several enhancement pipelines side by side on multi-channel input. The candidate set has six arms:

  • MVDR beamforming with three covariance forgetting speeds (fast, medium, slow).
  • An ARiSE-style auto-regressive enhancer, or a reimplementation if code is unavailable.
  • A single-channel neural mask enhancer.
  • An unprocessed reference microphone.

Every block of about two seconds, the learner observes the context $x_t$ and plays one output. It then receives a reward only for that output.

Feedback comes from simulated listeners. Each listener is a hidden weighted combination of MAPSS Perceptual Match and Perceptual Separation computed against the reference, passed through a noisy pairwise or thumbs-up response. Weights are drawn to span listeners from leakage-averse to distortion-averse. RealMAN ships an estimated direct-path clean reference and SonicSet has exact references, so intrusive MAPSS rewards are available on both real and simulated streams. We also train a non-intrusive predictor of these scores on SonicSet to use as a feature and as a baseline.

The learner is thresholded Lasso greedy, one sparse model per arm, over a context of roughly 1,000 dimensions. The context combines spatial state estimates, spatial covariance summaries, and pooled early-layer wav2vec 2.0 features. The analysis tracks the quantity the theory says governs success: the smallest eigenvalue of each arm's accumulated feature covariance on the rounds where that arm was chosen. We compare its growth rate across talker speeds using SonicSim, which lets us set trajectories from static to walking pace. The prediction is linear growth for moving talkers and stalled growth, with occasional permanent lock-in to a suboptimal arm, for static ones.

Baselines

The naive reference is the best single fixed pipeline, chosen offline on SonicSet validation data and never changed. It represents what a device ships today. Any adaptive method must beat it on the listener's own reward.

The strongest non-learning competitor is predictor best-of-K. It computes the non-intrusive quality predictor on all six parallel outputs and plays the highest-scoring one. It needs no feedback at all and exploits the fact that all outputs are computed anyway. It gets the same features and the same predictor training data, so the only thing it lacks is information about the individual listener. Beating it means lower cumulative regret against listener-specific rewards within a realistic feedback budget.

The strongest explicit-exploration competitors are sliding-window LinUCB and discounted linear Thompson sampling, the standard answer to drift. The trade-off here is known: explicit exploration pays off only when contexts fail to span the feature space. Both use identical features, and exploration strength and window length are tuned on SonicSet validation with the same budget as our learner's thresholds. We also run epsilon-greedy and plain least-squares greedy without sparsity, to separate the effect of greediness from that of high-dimensional support recovery.

The key ablation removes the spatial-state block $s_t$ from the context and instead gives the greedy learner a forgetting window, so motion appears as parameter drift. If our claim is right, this variant loses both regret and its diversity advantage.

No reported result was found on Lacuna for online regret, or for per-listener adaptive selection, on RealMAN or SonicSet. For the component pipelines, the reference figures are the ones ARiSE reports on its six-microphone simulated benchmark at -5 dB SNR: PESQ 2.39 and ESTOI 0.806, versus 2.05 and 0.726 for its IGCRN baseline. We will reproduce the fixed-arm numbers on that setup before running any bandit comparison.

Why not just…?

Why not just use sliding-window LinUCB or Thompson sampling? Local Anti-Concentration Class and Thresholded Lasso Bandit both find that explicit exploration wastes rounds when contexts are already diverse. The bonus is especially wasteful in high dimensions, where it inflates along irrelevant features. In a listening device, every exploratory round is also audibly bad audio. Still, with static or slowly moving talkers, explicit exploration may genuinely win, and we expect it to. The speed sweep will locate the crossover. We report exploratory-round audio quality separately, so the cost of exploration is visible rather than hidden in averages.

Why not just score every output with a quality predictor and play the best one? This is the most serious threat. It needs no online learning, and all six outputs exist anyway. It fails only to the extent that individual listeners deviate from the predictor's average notion of quality. We therefore vary how much simulated listeners differ in their leakage-versus-distortion weighting. If listeners are near-homogeneous, we expect best-of-K to match or beat the bandit, and we will report that plainly. The bandit also receives the predictor's scores as features, so it learns only the listener-specific correction. It should never be much worse than best-of-K and should pull ahead as listener heterogeneity grows.

Why not just train one large offline model on SonicSet? A single TF-GridNet-style model trained on SonicSim data transfers better than Libri2Mix-trained models. It still encodes one average preference and suffers the simulation-to-real gap documented by RealMAN, where localization falls from 89.2% to 52.4% in their reported comparison. Such a model does not compete with our learner; it is one of the arms. The question is whether choosing among it and cheaper pipelines per listener and per condition beats always using it, which the fixed-arm baseline measures directly.

Why not just handle motion with a forgetting window on the parameters? Forgetting discards exactly the data sparse support recovery needs. It also treats the predictable effect of position as unexplained drift. The spatial-state ablation described above tests this directly.

Proposed Experiments

Diversity and lock-in on simulated trajectories. On SonicSet trajectories at controlled speeds, including a static condition, we measure three things: cumulative regret over horizons of several thousand blocks, the rate at which each arm's minimum covariance eigenvalue grows, and the fraction of seeds in which greedy locks permanently onto a suboptimal arm. The decisive check is that lock-in frequency is high for static talkers and falls toward zero as speed rises. Explicit-exploration baselines should show the reverse ordering of advantage.

Dimension scaling. We append increasing numbers of embedding dimensions and pure-noise coordinates, following the Yahoo! R6A padding protocol of Thresholded Lasso Bandit. We then check whether thresholded greedy's regret grows roughly with $\log d$ while unsparsified greedy and LinUCB grow much faster. We also measure how far correlated speech embeddings erode this, given that the theory's balanced-covariance assumption likely fails.

Real moving talkers. On RealMAN moving-source recordings, with rewards computed from the provided direct-path references, we repeat the comparison against all baselines. This checks whether real trajectories carry enough diversity or whether simulation overstates it.

Reward integrity and objective-metric divergence. For every method we also report SI-SDR, PESQ and ESTOI. We look specifically for cases where the learner's choices raise the perceptual reward while lowering signal metrics, or the reverse. On a small held-out set we run a listening test in the MUSHRA style used by SEBASS. It checks that the MAPSS-driven choices are the ones human listeners actually prefer, and that selection among fixed pipelines has not found an artifact the metric likes but listeners do not.

The proposal succeeds if moving talkers yield sublinear, low-regret greedy learning with rare lock-in. Greedy must beat tuned explicit exploration at walking speeds, and it must beat predictor best-of-K once listeners are heterogeneous. It fails informatively if real trajectories do not span the feature space, which would indicate that deployed adaptive audio still needs deliberate exploration.