When a voice call drops packets, the receiver must play something in the gap. Modern neural concealers fill short gaps well. On long bursts they face a choice that no current system makes deliberately. The concealer can keep generating plausible speech, which risks inventing words the speaker never said. Or it can fade to comfort noise, which sounds like a dropout but tells the listener no lies. The right choice depends mostly on how long the burst will last and on what was actually said, and the receiver knows neither while the gap is happening. The training pipeline, however, knows both: it holds the recorded loss trace and the clean reference audio.
We propose to learn this decision with an asymmetric actor-critic. This is a reinforcement-learning setup in which the deployed policy (the "actor") sees only what is available at run time. A second network (the "critic") scores the actor's choices during training using privileged information, and is then thrown away. The actor is a tiny causal controller placed on top of a frozen neural concealer. Every 20 ms it chooses whether to continue generating, attenuate, switch to comfort noise, or set how to crossfade back when real packets resume. The critic sees the future of the loss trace and the clean speech that was lost. The claim is that this separation lets a policy with no lookahead approach the quality of one that knows the future. It should do so specifically on the long bursts where current systems degrade.
Core Claim
Existing concealers treat the problem as supervised inpainting, meaning they learn to reconstruct the missing waveform frame by frame. The strongest line of work already exploits privileged future audio. BS-PLCNet 2 trains a causal encoder path to mimic a non-causal path that sees the future, then discards the non-causal path at inference. That is distillation of waveform representations, not a decision policy. By its own authors' account it still degrades on bursts beyond about seven packets (over 140 ms), where regressing toward a future-aware teacher produces muffled output.
Our position is that the long-burst failure is not a reconstruction problem that more capacity will fix. It is a decision problem under partial observability. The receiver has the same causal history whether the burst will end in 60 ms or in 600 ms, yet the best action differs sharply between those cases. The asymmetric actor-critic literature calls this value aliasing: states that look identical to the actor lead to very different returns. That literature shows a critic given the hidden information can resolve it without making the deployed policy depend on that information.
What would be new is a formulation of concealment control as a POMDP (a decision process where the true state is hidden from the agent). In it, the actions are discrete concealment decisions rather than audio samples, and the reward is the challenge metric computed on whole utterances. The critic follows the unbiased history-state form of Baisero and Amato: it is conditioned on the actor's history and on the privileged state. It also follows the privileged-signal selection of Informed Asymmetric Actor-Critic.
We deliberately do not use reinforcement learning to generate waveforms. Where the target waveform is known and losses are differentiable, supervised training gives exact dense gradients, and replacing it with policy gradients would only add noise. RL earns its place only for the discrete, long-horizon choices that no waveform loss supervises.
Research Program
Generator. The generator is a frozen causal neural concealer. We retrain it to the published recipe of End-to-End Multi-Loss Training for Low Delay Packet Loss Concealment: a time-domain Wave-U-Net with 20 ms total latency, 2.36M parameters, and a real-time factor of 0.27. We use it because it is the first-place system of the INTERSPEECH 2022 challenge with fully specified training and latency. We then repeat the key comparison with a band-split generator in the style of BS-PLCNet, to check that the controller is not specific to one generator.
Controller (the actor). A small recurrent network, on the order of tens of thousands of parameters, runs once per 20 ms frame. It receives only causal signals: the arrival pattern of recent packets, the elapsed length of the current gap, the energy and voicing of recent received audio, and a summary of the generator's recent output. Its action set is small. It can continue generative concealment, apply attenuation at one of a few rates, or switch to comfort noise. At burst end it chooses among a few crossfade lengths for rejoining real packets, the point where phase clashes (Time-warping and re-phasing in packet loss concealment) produce clicks. The controller adds no lookahead, so the 20 ms latency budget is untouched.
Reward. The reward is assigned once per utterance:
$$R = \mathrm{PLCMOS}(\hat{x}) \;+\; \lambda_1\,\mathrm{WAcc}(\hat{x}, y) \;-\; \lambda_2\,\mathrm{Ins}_{\text{gap}}(\hat{x}, y).$$
Here $\hat{x}$ is the played-out audio and $y$ the clean transcript. PLCMOS is the challenge's neural estimate of human opinion score, and WAcc is ASR word accuracy. $\mathrm{Ins}_{\text{gap}}$ counts ASR insertions and substitutions whose timestamps fall inside lost regions; this is our operational measure of hallucinated words. The weights $\lambda_1, \lambda_2$ are fixed before training by matching the challenge's final-score weighting, with $\lambda_2$ swept only on validation data. Training uses Whisper as the ASR. Evaluation uses a different ASR (WeNet) and non-reward metrics, so that the policy cannot simply learn to exploit one recognizer.
Critic. The critic takes the actor's history $h_t$ together with privileged signals $s_t$ drawn from the training trace and reference audio. Candidate privileged signals are the remaining length of the current burst, whether the lost segment contains speech or silence, and a frozen wav2vec 2.0 embedding of the clean lost segment. Following Informed Asymmetric Actor-Critic, we score each candidate before training by residual informativeness: an independence test (HSIC) of whether the signal predicts return beyond what $h_t$ already predicts. We include only signals that pass.
The critic must condition on $h_t$ as well as $s_t$. What the generator has already emitted is part of the actor's history, and it changes the value of continuing even when the clean future is the same. A state-only critic therefore carries exactly the history-aliasing bias that Baisero and Amato prove. They show that bias causes collapse in memory-dependent tasks. A Theoretical Justification for Asymmetric Actor-Critic Algorithms supplies the reason to expect lower suboptimality: privileged conditioning removes the aliasing and inference error terms from the bound. It also warns that a richer critic input can raise approximation error, which is one more reason to keep $s_t$ small. Sinha and Mahajan find that auxiliary history-compression losses become redundant under a privileged critic. We therefore add no representation-learning losses to the actor, and we test that choice as an ablation.
Training setup. Episodes are utterances paired with real loss traces from the challenge training split. The generator's outputs can be cached per action branch because the action set is small, so rollouts are cheap. We use PPO-style updates.
Baselines
Naive reference points. These are zero-filling and the generator run with no controller, meaning it conceals every gap to the end. For zero-filling, the INTERSPEECH 2022 challenge reports a final score of 0.73. End-to-End Multi-Loss Training reports 1.58 PLC-MOS at 30–50% loss.
Strongest existing system. The uncontrolled generator is itself the strongest existing competitor on this question, because it is the published first-place system. It reports blind-test PLC-MOS 4.282, word accuracy 0.875, and final score 0.845 in End-to-End Multi-Loss Training. The challenge overview lists the winning team's final score as 0.98. We read this as a different normalization or evaluation stage and will report our numbers against both, computed with the official scripts, rather than guess which applies.
For the ICASSP 2024 blind set, the reference is BS-PLCNet 2. It reports a 0.18 PLCMOS gain over BS-PLCNet, whose reported PLCMOS is 4.02 with WER 15.71%. BS-PLCNet 2 is the most direct prior use of privileged future audio in concealment.
What counts as beating these systems. Beating them does not require a higher overall score, since most gaps are short and the controller should mostly leave them alone. It means that on bursts longer than 140 ms the controlled system raises PLCMOS and lowers in-gap insertions under the held-out ASR. At the same time it must not lower the overall final score, with differences judged by confidence intervals over utterances. All systems share the same generator weights, the same training traces, and the same evaluation scripts.
Hand-tuned rule. This is a codec-style rule that attenuates after $N$ lost frames, with $N$ and the attenuation rate grid-searched on validation using the same reward. It tests whether any learning is needed at all.
Two learned controllers from the same data. Both have the same architecture and the same budget of utterance rollouts as our controller. The first imitates an oracle expert that sees the future loss trace and picks the best action per burst by exhaustive search. The second first predicts the remaining burst length, then applies the thresholds of the oracle rule.
Critic ablations. These isolate the key ingredient. We compare four critics, each with equal parameter count and tuning budget: symmetric $V(h)$, state-only $V(s)$, history-state $V(h,s)$ with all candidate privileged signals, and history-state with HSIC-selected signals.
Why not just…?
Why not just use BS-PLCNet 2's non-causal distillation, which already exploits the future? Distillation teaches the causal path to match what a future-aware path would compute. Under ambiguity, the best such match is an average over possible futures, which is precisely the muffled long-burst output its authors report. Distillation never asks whether continuing to generate was worth it. We will run our controller on top of a BS-PLCNet-style generator. If distillation had already resolved the long-burst decision, the controller would learn to always continue and gain nothing, which is a clean falsification.
Why not just imitate an oracle that knows the burst length? This is the most serious threat, because it is cheaper and more stable than RL. Its weakness is that the oracle acts on information the actor lacks. Cloning it trains the actor to reproduce actions it cannot justify from its own history, and under aliasing that yields a policy averaged across the oracle's cases. That average need not be the best causal policy. The asymmetric critic instead optimizes expected return given the actor's information. The oracle-imitation baseline is run with equal data. If it matches our controller on long bursts, we will report that the asymmetric critic is unnecessary here.
Why not just predict the remaining burst length and threshold it? A predict-then-act pipeline is interpretable and may capture most of the gain if burst length is the only hidden variable that matters. It cannot weigh what the lost speech contained, or how the generator's output so far affects the value of continuing. The critic can, through the wav2vec and history inputs. The comparison, split by whether the lost segment is speech or silence, shows whether that extra information pays off.
Why not just let the generator learn to fade, or train it with a MetricGAN-style quality discriminator? Discriminator-trained generators like PLCNet and BS-PLCNet already optimize perceptual quality. However, their losses compare generated audio to reference audio frame by frame. They never price the utterance-level trade-off between a fluent wrong word and an honest dropout. Our in-gap insertion metric measures that trade-off directly for every system.
Proposed Experiments
Data and evaluation. We train on the INTERSPEECH 2022 challenge training audio and its real Microsoft Teams loss traces. We evaluate on its blind test set with the official PLCMOS and final-score scripts. We transfer without retraining to the ICASSP 2024 blind set used by BS-PLCNet.
As a long-burst stress test, we use VCTK under the Markov-chain "Hard" loss setting described in TD-PLC: 45% loss with bursts of 400–700 ms. TD-PLC reports PLCMOS 4.08 there, but it transmits extra semantic side information, so its number is context rather than a target.
Every result is broken down by burst length, with buckets at 20–60, 60–140, 140–400, and above 400 ms. Metrics are PLCMOS, PESQ, word accuracy under the held-out ASR, and in-gap insertions.
Check 1: does the controller help where predicted? Gains should concentrate above 140 ms and be near zero below 60 ms. Uniform gains would suggest the controller is exploiting the metric rather than solving the aliasing problem. We will check that possibility against PESQ, which is not part of the reward, and against a small crowd-sourced CMOS listening test on the long-burst bucket.
Check 2: does the privileged critic matter? For this check we compare critic variants on learning curves (reward against rollouts) and final long-burst score across five seeds. The prediction from the aliasing theory is ordered: history-state beats symmetric. A second prediction is specific: the state-only critic should be unstable or worse on utterances where the generator has already produced audio in the gap, which is where history aliasing bites.
Check 3: does signal selection matter, as the "Noisy TV" result predicts? Here we compare HSIC-selected against all-signals critics. A null result is informative: it would mean that for audio the wav2vec embedding is not structured noise, contrary to the general finding.
Check 4: does the result transfer across generators? We run the full comparison with both generator families. We report that the controller's added compute leaves the real-time factor and the 20 ms algorithmic latency unchanged.
References
- BS-PLCNet 2: Two-stage Band-split Packet Loss Concealment Network with Intra-model Knowledge Distillation
- Unbiased Asymmetric Actor-Critic for Partially Observable Reinforcement Learning
- Informed Asymmetric Actor-Critic: Leveraging Privileged Signals Beyond Full-State Access
- End-to-End Multi-Loss Training for Low Delay Packet Loss Concealment
- Bs-Plcnet: Band-Split Packet Loss Concealment Network with Multi-Task Learning Framework and Multi-Discriminators
- Time-warping and re-phasing in packet loss concealment
- A Theoretical Justification for Asymmetric Actor-Critic Algorithms
- Asymmetric Actor-Critic with Approximate Information State
- INTERSPEECH 2022 Audio Deep Packet Loss Concealment Challenge
- PLCNet: Real-time Packet Loss Concealment with Semi-supervised Generative Adversarial Network
- TD-PLC: A Semantic-Aware Speech Encoding for Improved Packet Loss Concealment