Audio super-resolution, also called bandwidth extension, means generating the missing high frequencies of a recording sampled at a low rate (16 or 24 kHz) so that it plays back at studio rate (48 kHz). The best generalist upsampler, AudioSR, is a latent diffusion model, a generator that builds its output by removing noise over many steps. Its authors say it is too slow for real-time use. Fast upsamplers exist: the 43M-parameter cascade MS-BWE and the single-step FLowHigh. Both are trained only on clean studio speech (VCTK) that has been artificially low-pass filtered. The natural way to give a fast model the slow model's breadth is distillation. The slow teacher upsamples large amounts of real low-rate audio, and the fast student is trained on those outputs, which act as pseudo-labels.
Two costs make this naive. The teacher's high bands are sometimes wrong, with metallic or buzzing artifacts, and a student that regresses onto them learns the errors. The usual remedy is to estimate how reliable each label is through ensembles or repeated sampling, as in the 18-model consensus filter of Auto-Annotation Quality Prediction. That remedy multiplies the teacher cost that was already the bottleneck. Our claim is that bandwidth extension carries a cheap, built-in check on itself. Take a clip whose real content stops at cutoff $c$. Low-pass it further to $c/2$ and ask the teacher to restore the band from $c/2$ to $c$, where the truth is known. The teacher's measured error on this held-out band, frame by frame, predicts how wrong its pseudo-label is above $c$, where the truth is unknown. We use that predicted error to weight the student's training loss. The cost is one extra teacher pass, it needs no ensemble, and the predictor can be validated directly.
Core Claim
The claim has two parts. First, on audio where the full 48 kHz truth exists, the teacher's per-frame error on the restored held-out band correlates with its per-frame error on the real high band at the same upsampling ratio. Second, a student trained on real low-rate audio with losses down-weighted by this score will match or beat a student given the same teacher compute spent on more uniformly weighted pseudo-labels or on repeated-sample variance. The comparison is made on out-of-domain audio, and the gain must exceed what the student gets from clean paired data alone.
The mechanism rests on keeping the upsampling ratio the same. A 16 kHz clip (cutoff 8 kHz) is checked by restoring 4 to 8 kHz from a 4 kHz-cutoff copy. A 24 kHz LibriTTS clip (cutoff 12 kHz) is checked by restoring 6 to 12 kHz. Each frame's held-out-band log-spectral error $e_t$ becomes a weight
$$ w_t = \exp\!\left(-\frac{\tilde e_t}{\tau}\right), $$
where $\tilde e_t$ is $e_t$ smoothed over neighbouring frames and normalised within each recording, and $\tau$ is a temperature fixed on a held-out set. The student's high-band reconstruction loss on frame $t$ is multiplied by $w_t$. Its adversarial and feature losses are left unweighted, so that uncertain frames still teach the student what realistic high frequencies sound like.
This borrows the soft, uncertainty-weighted loss of Combating Noise. That paper weights each region's loss by its uncertainty in a single pass, and showed that hard confidence cutoffs fail where errors cluster near decision boundaries. Its method was demonstrated only for 2D object detection. What we change is where the uncertainty comes from. We do not read it off the model's confidence, which is not available for a continuous spectrogram output. We measure it against held-out signal.
Relation to Prior Work
The upsampling papers motivate the gap and supply the components. NU-Wave, FLowHigh and MS-BWE all train on synthetic low-pass pairs of clean studio speech. None of them uses unlabeled real recordings. AudioSR is broad but slow, and it defers distillation to future work. We use AudioSR as the teacher and MS-BWE's architecture as the student.
AudioSR and FLowHigh overwrite the generated low band with the input. That trick makes a naive consistency check meaningless, because a teacher's output always agrees with its input below the cutoff. Our check withholds real signal from the teacher instead of comparing the output against what the teacher was given.
The distillation literature supplies the overall recipe and the cost argument. UDAPDR distils noisy synthetic text queries through ensembles of rerankers. Token-Level Ensemble Distillation uses 10 teachers to soft-label discrete phonemes. Tri-net accepts a pseudo-label only if repeated dropout passes agree. All of these estimate reliability through redundancy and act on discrete outputs. We estimate reliability through a held-out measurement on continuous spectra.
Beyond Quantity constrains the design. It shows that adding more synthetic data saturates early, so "just label more clips" is not assumed to win.
Research Program
The first stage validates the predictor before any student is trained. We take 48 kHz audio with full truth: VCTK held-out speakers (the MultiSpeaker split of NU-Wave and FLowHigh), and the singing corpus GTSinger, whose release sampling rate we will verify. On each clip we run AudioSR twice. One run starts from the $c$-cutoff input and is scored on the true high band. The other starts from the $c/2$-cutoff input and is scored on the held-out band. We then measure the rank correlation between the two error measures, at the frame level and the clip level. The check should be informative on each content type: voiced speech, fricatives, singing, and environmental sound.
The second stage is controlled distillation. We take 48 kHz corpora that the teacher did not train on: the GTSinger test portion and held-out VCTK speakers. We hide their high bands, treat them as unlabeled pseudo-label pools, and keep the truth only for evaluation. Every student uses the same MS-BWE-style architecture and the same clean VCTK paired data. The only thing that varies between students is how the pool is used.
The third stage repeats the recipe on audio that genuinely lacks high frequencies. The pool is LibriTTS at 24 kHz (LibriTTS), from which we extend 12 to 24 kHz. Here there is no truth above 12 kHz, so we judge quality by listening tests.
Baselines
The do-nothing reference is the student trained only on clean VCTK synthetic pairs, following the MS-BWE recipe. That paper reports 1271.81x real-time on an RTX 4090 and 59.70x on CPU, and says it outperforms NU-Wave2 and UDM+ on Log-Spectral Distance (LSD) and on ViSQOL, a model of perceived quality. The file gives no exact LSD figures for it, so we retrain it ourselves rather than quote a number.
The quality ceiling is AudioSR itself, run at full sampling steps on the same test clips. The file contains no reported LSD or MOS values for AudioSR. FLowHigh claims better LSD, high-band LSD and ViSQOL than AudioSR from 8, 12, 16 and 24 kHz inputs, but the file gives no exact values. As a calibration point we report NU-Wave's SingleSpeaker r=2 LSD of 0.810 (NU-Wave).
The strongest competitor to our weighting is equal-compute redundancy. Our method spends two teacher passes per clip. The redundancy baseline spends the same two passes on two independent AudioSR samples and weights each frame by the spread between them, which is the cheapest form of the ensemble idea in Auto-Annotation and Tri-net. A second equal-compute baseline spends both passes on twice as many clips, each labeled once with uniform weights. A no-teacher baseline trains the student on the pool's own known band at the same ratio, for example 4 to 8 kHz from a 4 kHz cutoff, so it adapts to the domain without any pseudo-labels.
Two ablations isolate the key ingredient. One uses the same held-out-band score as a hard filter, dropping the worst frames instead of soft-weighting them. The other replaces the score with a random score of matched distribution.
All students get the same number of training steps, the same tuning budget of identical sweeps over learning rate and $\tau$-equivalent parameters, and the same total teacher GPU-hours. Beating the strongest baseline means two things on the hidden-truth out-of-domain test: lower high-band LSD and higher ViSQOL than the equal-compute redundancy and double-data students, with confidence intervals from three seeds. It also means a significant preference in a listening test (MUSHRA-style), because AudioSR documents that LSD can improve while metallic artifacts get worse.
Why not just…?
Why not just train the fast student on more clean paired data and skip pseudo-labels? This is the most serious threat. MS-BWE and FLowHigh already do well on VCTK with clean pairs alone. If the do-nothing student matches distilled students on GTSinger and the other out-of-domain sets, pseudo-labeling is not worth its cost and we will report that. The controlled stage-two test exists to settle this, and the in-domain VCTK result alone is not counted as evidence.
Why not sample the teacher several times and use the spread as uncertainty? Spread between generative samples mixes real error with legitimate ambiguity: many high bands are plausible for one low band. Large spread therefore does not reliably mark a wrong pseudo-label, and each extra sample costs a full diffusion run. This baseline is run at equal compute, and stage one also measures how well the spread correlates with true high-band error, alongside our score.
Why not just drop low-quality clips with a hard threshold? Combating Noise found hard cutoffs harmful exactly where errors concentrate. Also, a whole-clip filter wastes the good frames of mostly-bad clips, which is the "filtration tax" of discarded generation compute. The hard-filter ablation tests this directly with the same score.
Why not adapt to the domain using only its known band, without a teacher? Training at a lower ratio on in-domain audio is free, but it never shows the student real-domain content above $c$. Which effect dominates is an empirical question, and the no-teacher baseline answers it.
Decisive Checks
The idea fails at stage one if the held-out-band error correlates weakly with true high-band error, for example a Spearman correlation below about 0.3 at the frame level. That is plausible for content such as sibilants, whose energy sits mostly above 8 kHz. In that case we would report the band-wise correlation map as the main finding and stop. The idea succeeds if, at matched teacher compute, the weighted student beats both the redundancy and double-data students on out-of-domain high-band LSD, ViSQOL and listener preference. The random-score ablation must also lose most of that gain. The LibriTTS 12-to-24 kHz listening test then checks whether the effect carries over to audio that never had high frequencies.
References
- Audiosr: Versatile Audio Super-Resolution at Scale
- MultiStage Speech Bandwidth Extension with Flexible Sampling Rate Control
- FLowHigh: Towards Efficient and High-Quality Audio Super-Resolution with Single-Step Flow Matching
- Auto-Annotation Quality Prediction for Semi-Supervised Learning with Ensembles
- Combating Noise: Semi-supervised Learning by Region Uncertainty Quantification
- NU-Wave: A Diffusion Probabilistic Model for Neural Audio Upsampling
- UDAPDR: Unsupervised Domain Adaptation via LLM Prompting and Distillation of Rerankers
- Token-Level Ensemble Distillation for Grapheme-to-Phoneme Conversion
- Tri-net for Semi-Supervised Deep Learning
- Beyond Quantity: Trajectory Diversity Scaling for Code Agents
- GTSinger: A Global Multi-Technique Singing Corpus with Realistic Music Scores for All Singing Tasks
- LibriTTS: A Corpus Derived from LibriSpeech for Text-to-Speech