Few-Step Blind Dereverberation by Refitting the Room at Every Step

Blind dereverberation means recovering clean speech from a recording made in an unknown room, without knowing that room's echo pattern (its room impulse response). The most robust recent methods treat it as a joint problem. A diffusion model, which is a generator that learns to turn random noise into speech by denoising it step by step, supplies a prior over clean speech. A small physical model of the room is fitted alongside it, and the two estimates correct each other. This joint approach generalizes to rooms never seen in training, where supervised networks trained on paired data break down. It is also far too slow for real use, because every one of hundreds of sampling steps carries an extra room-fitting update.

We propose to run this joint inference in two to four steps. The approach rests on one observation. A consistency model, which is a network distilled to jump from any noise level straight to a clean estimate, produces a usable clean-speech estimate at every step. The noisy intermediate guesses that slow diffusion samplers rely on are no longer needed. Given a nearly clean speech estimate, refitting a low-dimensional room model (plus an additive-noise level) is a small, well-conditioned least-squares problem. Re-imposing consistency with the recording under the refitted room can then be done in closed form in the frequency domain. Each step therefore becomes one network evaluation, one cheap room refit, and one closed-form correction. The research question is whether this keeps the mismatched-room robustness of the slow joint methods. It must also beat the obvious shortcut: predict the room once, then run a fast solver that assumes the room is known.

Core Claim

In blind single-channel dereverberation with additive noise, alternating room refitting with a consistency-model speech prior over two to four steps can match the quality of full-trajectory joint diffusion methods in mismatched rooms, at roughly two orders of magnitude less compute. The same method should outperform a "predict the room once, then restore" cascade, and the margin should be largest precisely when the room-predictor's initial estimate is wrong.

Where This Sits

The joint side of the problem is well developed but slow. BUDDy and its extended version, Unsupervised Blind Joint Dereverberation and Room Acoustics Estimation with Diffusion Models, pair an unconditional speech diffusion prior trained on VCTK with a subband exponential-decay room model. They alternate a posterior-sampling speech update with a gradient update of the room parameters across the reverse trajectory, warm-started from Weighted Prediction Error, a classical linear-prediction dereverberation method. We reuse their room model and their unsupervised setting. Those papers name their computational cost as the main limitation, and they also omit additive noise; we target both.

Two other methods reduce the cost of joint inference but stay multi-step. Fast Diffusion EM interleaves one blur-kernel update per step of a single reverse trajectory, cutting blind deblurring from about 90 s to about 9 s on an A100. BlindDPS runs parallel diffusion processes for the image and the operator, which doubles the network evaluations per step. Both depend on full trajectories and on Tweedie-style posterior-mean estimates. Those estimates are poor at coarse step sizes, which is exactly what makes a few-step regime hard.

The fast side of the problem is equally developed but non-blind. CoSIGN guides a frozen consistency model with a ControlNet (a trainable side network for conditioning) and a hard projection step, solving inverse problems in one to two steps. Learnable Linear Extrapolation shows that data-consistency corrections knock few-step samplers off their trajectory and repairs this with learned step weights. Inverse Flow and Consistency Models extends consistency training to general known noise processes and explicitly leaves the blind case open. All three assume a known forward operator. We supply the missing piece: an operator estimate that is re-fitted inside the few-step loop.

Two further papers constrain the design. ROSE-CD reaches one-step speech enhancement, but its perceptual loss trades off waveform fidelity. That warns against optimizing PESQ at the expense of consistency with the measurement, so we keep the measurement constraint explicit. Diffusion Posterior Sampling for Informed Single-Channel Dereverberation shows that naive regularized inverse filtering produces pre-echo and collapses under noise. Our closed-form correction must therefore be a noise-aware blend with the prior's estimate, not a direct inverse.

Research Program

The first component is a consistency model of clean speech. We obtain it by consistency distillation of an unconditional speech diffusion prior trained on clean VCTK, so the pipeline never sees reverberant-clean pairs.

The second component is the room model. We use the subband exponential-decay parameterization of BUDDy, with parameters $\theta$ (per-band initial level and decay rate), and add a per-band additive noise power $\sigma^2$.

Given the consistency model's clean estimate $\hat{x}_k$ at step $k$ and the recording $y$, the room refit solves

$$ \hat\theta_k, \hat\sigma_k^2 = \arg\min_{\theta,\sigma^2} \; \sum_{f,t} \frac{|Y_{f,t} - (H_\theta \hat X_k)_{f,t}|^2}{\sigma_f^2} + \log \sigma_f^2 , $$

where $Y$ and $\hat X_k$ are short-time Fourier transforms, $f$ and $t$ index frequency and frame, and $H_\theta$ is subband convolution with the parametric room response. Because $\theta$ has only a few dozen dimensions and $\hat{x}_k$ is already nearly clean, a handful of Gauss–Newton iterations should suffice.

The correction step replaces gradient guidance with a per-bin Wiener-style combination. It merges the prior's estimate with the deconvolved recording, weighted by the estimated noise and by the prior's uncertainty at the current noise level. The result is then re-noised to the next, smaller noise level, following the usual multistep consistency sampling.

The third component guards against collapse. Blind deconvolution admits a degenerate solution: an identity room with reverberant "speech." The initial room estimate comes from a small regressor that predicts $\theta$ from the recording. The same regressor serves as the cascade baseline, which makes the ablation clean. The clean-speech prior and the decay parameterization jointly penalize the degenerate solution. Because the operator changes between steps, we will also test whether extrapolation weights in the style of LLE, learned on about 50 held-out examples, stabilize the two-to-four-step trajectory. This addresses the gap left by LLE's fixed-operator setting.

Baselines

The naive reference point is the reverberant, noisy input itself, together with Weighted Prediction Error alone. These set the floor on every metric and on runtime.

The strongest quality competitor is full-trajectory BUDDy BUDDy. It will run with its published step count and warm start on identical test mixtures and hardware. The same pretrained speech prior will serve as the teacher for our consistency model, so the difference between the two isolates sampling strategy rather than prior quality. The file reports BUDDy's DNS-MOS as 3.74 in the mismatched VCTK setting. Matching that within a small tolerance at two to four steps counts as success. No reported runtime for BUDDy was found on Lacuna, so we will measure its real-time factor ourselves.

The strongest latency competitor is a cascade. The room regressor predicts $\theta$ and $\sigma^2$ once, and then the identical consistency model and Wiener correction run with the room frozen. This non-blind few-step inference follows CoSIGN's hard-constraint idea CoSIGN. It is also exactly our method with the refit switched off, which makes it the ablation that isolates the key ingredient. To beat it, we must show a statistically significant improvement in SI-SDR and in room-parameter error on mismatched and real rooms, at equal step count and within a modest runtime overhead.

A multistep interleaved baseline adapts Fast Diffusion EM's one-update-per-step scheme to the room model. It will be run with a standard fast sampler (DDIM) at 4, 10, and 30 steps. Its reported speedup, 90 s down to about 9 s on an A100 for image deblurring, frames what interleaving alone buys.

A supervised one-step baseline follows ROSE-CD ROSE-CD. A consistency-distilled enhancer is trained on paired reverberant data from simulated rooms, with the same simulation budget and tuning budget as our room regressor. ROSE-CD reports an RTF of 0.048 and a PESQ of 3.99 on VoiceBank-DEMAND, a denoising task, so these are latency targets rather than directly comparable quality numbers.

All learned components get the same simulated-room data and the same number of hyperparameter trials. All runtimes are measured on one GPU type.

Why not just…?

Why not just predict the room once and run a fast non-blind solver? This is the real threat. If the regressor is accurate, refitting adds nothing. The evidence against it in the file is indirect. BUDDy's papers show supervised models losing PESQ sharply in mismatched rooms, and scSplit shows that estimating the degradation severity explicitly matters: in its dominant-signal regime it reaches 40.9 PSNR against InDI's 33.1. Neither result proves that a one-shot room estimate is insufficient. Our decisive experiment settles the question directly. We will plot the joint method's gain over the cascade against the regressor's initial error in T60 and direct-to-reverberant ratio. If the gain is flat near zero even for badly initialized rooms, the joint refit is not worth it, and we will report that.

Why not just train a supervised one-step dereverberator? Such a network is the fastest possible option. BUDDy's papers report that supervised diffusion baselines such as NCSN++M and PSE degrade sharply under room mismatch, while the unsupervised joint method holds up. Our mismatched and real-recording splits test whether a ROSE-CD-style model inherits that fragility. It likely wins on matched rooms, and we expect to concede that case.

Why not just plug a fast ODE solver into BUDDy or Fast Diffusion EM? On CelebA-HQ 3-step inpainting, LLE reports that a standard corrector-based solver reaches only 16.64 dB PSNR, which learned extrapolation raises to 24.38 dB. CoSIGN reports that posterior-sampling guidance fails at one to two steps. Both results suggest that Tweedie-based room updates will be poor at coarse steps. The interleaved DDIM baseline at 4 and 10 steps measures this directly for the joint audio case.

Proposed Experiments

For simulated data, we follow BUDDy's matched and mismatched VCTK protocol, with rooms generated by pyroomacoustics at T60 from 0.4 to 1.0 s. We add white and ambient noise at 5 to 20 dB SNR, the regime BUDDy does not model. For WSJ0 at 10 dB SNR, informed posterior sampling holds PESQ near 2.5 even with the true room DPS for Informed Dereverberation, which gives an informed upper reference. For real data, we evaluate on DDS-DAPS-RealRecorded and on the noisy-reverberant condition of WHAMR!. Singing voice at 44.1 kHz from NHSS serves as a stress test.

We report PESQ, ESTOI, DNS-MOS, and SI-SDR, together with room-estimate error (T60 and direct-to-reverberant ratio against simulator ground truth) and real-time factor.

Three checks decide the proposal. First, at two to four steps, the joint method must stay near BUDDy's full-trajectory quality in mismatched rooms. Second, it must beat the frozen-room cascade by a margin that grows with the regressor's initial error. Third, SI-SDR must not fall while DNS-MOS rises, since that pattern would indicate the fidelity loss seen in ROSE-CD. Further ablations remove the noise term, remove the warm start (to measure how often the identity-room collapse occurs), and add or remove the learned extrapolation weights.