A diffusion model is a generator that learns to turn random noise into images by repeatedly removing a little noise at a time. To train one, you corrupt real images to many different noise levels and teach the network to undo each one. To make it generate a requested class, you also give it a class label. When labels are scarce, a common recipe is to let a classifier guess labels for the unlabeled images and train the generator on those guesses, called pseudo-labels. Some of those guesses are wrong. Two separate findings suggest when that matters. First, a wrong label barely hurts at heavy noise levels, where the image is mostly static and the label carries little information, but it hurts badly at light noise levels, where the network is drawing fine class-specific detail. Second, those light-noise levels are also the hardest denoising tasks and benefit from being learned last.
This note proposes to treat these two facts as one problem. The training procedure should decide, for each noise level and each unlabeled image, whether the pseudo-label is reliable enough to be used at that noise level. It should also move compute from heavy-noise to light-noise training only as fast as reliable labels for the light-noise regime become available. Concretely: use every pseudo-label, even shaky ones, when the image is buried in noise. Demand high classifier confidence before a label is allowed to steer near-clean denoising. Let the share of near-clean training grow as the classifier's labels improve, instead of on a fixed timer. If this works, semi-supervised conditional generators get the sample diversity of keeping all pseudo-labels and the class fidelity of keeping only confident ones, from a change to the training sampler and loss alone.
Core Claim
The harm a wrong pseudo-label does depends on the noise level. So the confidence threshold for admitting a pseudo-label should also depend on the noise level, and the schedule of noise levels seen during training should be paced by pseudo-label reliability. Our central prediction is that a noise-level-dependent admission rule beats any single, noise-level-independent confidence threshold in class-wise sample quality. It should also keep the overall quality obtained from using every pseudo-label.
Three papers motivate this. Label-Noise Robust Diffusion Models shows that, under fixed label noise, the score conditioned on a noisy label is a mixture of clean class-conditional scores. Its mixing weights collapse toward the marginal label distribution as noise grows, so label errors are mostly a low-noise problem. Its method, TDSM, corrects the loss with a transition matrix but still samples noise levels uniformly and assumes the label noise is fixed. Denoising Task Difficulty-based Curriculum for Training Diffusion Models shows that heavy-noise denoising is easy and near-clean denoising is hard. It trains easy-to-hard by grouping noise levels into clusters and adding the next cluster when the loss plateaus for a fixed patience. That gives DiT-B an ImageNet-256 FID of 22.96 versus 30.27 for vanilla training. It assumes clean labels, and its pacing hyperparameters are set by hand.
The third paper defines the regime. Diffusion Models and Semi-Supervised Learners Benefit Mutually with Few Labels introduces DPT. It trains a conditional diffusion model on classifier pseudo-labels, then retrains the classifier on generated images. It does this with a static noise schedule, no handling of label errors, and no feedback between its stages.
The two findings create a tension that neither paper confronts. The noise levels a curriculum would teach first are exactly where pseudo-label errors are harmless. The levels it teaches last are exactly where errors are destructive. That is the alignment we exploit.
Mechanism
Let $x$ be an unlabeled image with pseudo-label $\hat y$ and classifier confidence $c(x)$. Let $\lambda$ be the log signal-to-noise ratio of the corruption level; large $\lambda$ means a nearly clean image. Let $\kappa(\lambda)$ be how much the label matters at that level. We measure $\kappa(\lambda)$ cheaply as the accuracy above chance of a small classifier that reads noisy images, the same kind of time-dependent classifier TDSM trains, evaluated on the labeled set. During training, the pseudo-label is used as the condition only if its expected damage is under a budget $\varepsilon$:
$$ (1 - c(x))\,\kappa(\lambda) \le \varepsilon . $$
Otherwise the example is trained with the null label, the same unconditional slot used for classifier-free guidance. The image is never discarded. At heavy noise, $\kappa$ is near zero and almost every pseudo-label passes. Near clean, only confident ones do. One budget $\varepsilon$ replaces a per-resolution or per-stage threshold, and we tune it on a small labeled validation split.
The curriculum lives in the distribution over $\lambda$ used to sample training noise levels. We use the Laplace-shaped density of Improved Noise Schedule for Diffusion Training, which concentrates compute near $\lambda = 0$ and is a static importance sampler. We shift its center over training. The center sits at heavy noise early. It is advanced to the largest $\lambda$ at which at least a fraction $\rho$ of unlabeled images still pass the admission rule.
As the classifier is refreshed, confidence and accuracy rise and more images pass at high $\lambda$. The sampler then moves toward near-clean levels. This replaces the patience-and-cluster pacing of Kim et al. with a signal tied to label quality. In that respect it is the self-paced idea of Self-Paced Learning for Latent Variable Models, where the model's own state decides when harder examples enter, applied to noise levels instead of samples.
The classifier is refreshed at a few fixed points by retraining on labeled data plus generated images, as in DPT's third stage. Because the generator's near-clean levels were trained mostly on admitted labels, those generations should be cleaner. That closes a loop DPT leaves open.
Research Program
The work has three parts. The first is a measurement study, and the proposal depends on it. On CIFAR-10 with 4 labels per class, we train conditional diffusion models while injecting pseudo-label errors at a controlled rate into only one band of noise levels at a time. We then measure class-wise FID, which is FID computed per class and averaged as in TDSM, and Classification Accuracy Score, the accuracy of a classifier trained on generated images. This produces a harm-versus-noise-level curve for realistic, classifier-correlated pseudo-label errors rather than the symmetric synthetic noise TDSM studied. If this curve is flat, the idea fails, and we stop there.
The second part builds the gated, reliability-paced trainer on top of a reimplementation of DPT. The third tests it at scale on ImageNet-256 with 1 label per class using a DiT-B backbone, the scale at which Kim et al. report results. We also run a fixed-noise transfer test on Clothing-1M. That dataset has about 61.5% label accuracy, so it isolates the gating rule from the evolving-label loop.
Baselines
The naive reference is DPT as published: train on every pseudo-label, sample noise levels uniformly or with the cosine schedule, and use a static pipeline. We reimplement it at our backbone size and compute so all methods share data, pseudo-label source, iterations, and tuning budget. The published DPT numbers are 3.08 FID and 59.0% top-1 on ImageNet-256 with 1 label per class, and 4.68% error on CIFAR-10 with 4 labels per class. They come from a larger U-ViT setup, so we report them for context but compare against our own matched run.
The strongest single competitor is TDSM from Label-Noise Robust Diffusion Models, applied to pseudo-labels. Its transition matrix is estimated from classifier errors on the labeled validation split, and its compute is matched; it reports about 1.5x overhead, so we give our method the same wall-clock. TDSM reports a CIFAR-10 class-wise FID of 15.92 versus 30.45 for standard training under 40% synthetic noise, and FID 4.94 versus 6.67 on Clothing-1M. Beating it on Clothing-1M would show the gating rule is competitive even when labels do not evolve. On the semi-supervised benchmarks, we count a win as lower class-wise FID at equal compute and equal or better overall FID.
The curriculum competitor is Kim et al.'s cluster-and-patience schedule applied to DPT. It is tuned over a small grid of cluster counts and patience values with the same budget we give $\varepsilon$ and $\rho$.
Two ablations isolate the key ingredient. One keeps our pacing but uses a single noise-level-independent confidence threshold, swept over the same number of values. The other keeps our noise-level-dependent gating but uses a static Laplace sampler with no pacing.
Why not just…?
Why not just threshold pseudo-labels by confidence, as most semi-supervised methods do? A single high threshold discards supervision exactly where it is free: at heavy noise, where wrong labels cost little. It also skews the conditioned pool toward easy majority classes, a known failure of static thresholds. A low threshold lets errors into near-clean denoising. The constant-threshold ablation tests this directly. If its best setting matches ours on class-wise FID and overall FID, the noise-level dependence adds nothing, and we will report that.
Why not just run TDSM on the pseudo-labels? This is the real threat. TDSM already exploits the noise-level structure of label noise through its time-dependent classifier. Our case against it rests on two properties of pseudo-labels. Their errors are instance-dependent and correlated with the classifier's own confusions, so a single class-level transition matrix fits them poorly; Part-dependent Label Noise documents how far instance-dependent noise departs from class-conditional models. And the errors change every time the classifier is refreshed. If TDSM with a re-estimated matrix after each refresh matches us, then the curriculum is unnecessary and per-example gating is merely a cheaper equivalent. The matched-compute comparison settles this. We also test TDSM combined with our sampler, since the two may stack.
Why not just improve the classifier, or iterate DPT more rounds? Better pseudo-labels shrink the problem but do not remove it at 1 label per class, where label errors are unavoidable. Extra DPT rounds feed the generator's own mistakes back into the classifier, the confirmation-bias loop described in Tri-net for Semi-Supervised Deep Learning. We give the DPT baseline the same number of classifier refreshes as our method. That makes any gain attributable to how labels enter the generator, not to how many times they were recomputed.
Why not just use the existing timestep curriculum? Kim et al. pace by loss plateaus, which say nothing about whether the labels guiding near-clean denoising are trustworthy. Their clean-label setting never needed that signal. The curriculum baseline tests whether reliability-based pacing, rather than easy-to-hard ordering alone, is responsible for the gain.
Decisive Checks
The first check is the measurement study. Pseudo-label errors confined to the heavy-noise band must cost clearly less class-wise FID than the same errors confined to the near-clean band. Otherwise the gating has nothing to exploit. The second is the constant-threshold ablation on CIFAR-10 with 4 labels per class and on ImageNet-256 with 1 label per class. Our rule must lower class-wise FID without raising overall FID. The third is classifier accuracy after the generated-image refresh: images from the gated generator should yield a better classifier than images from our matched DPT run, which tests whether the loop actually closes.
We also report wall-clock throughout. Estimating $\kappa(\lambda)$ on the labeled set at each refresh should cost far less than TDSM's per-step classifier. If it does not, we will say so.
References
- Label-Noise Robust Diffusion Models
- Denoising Task Difficulty-based Curriculum for Training Diffusion Models
- Diffusion Models and Semi-Supervised Learners Benefit Mutually with Few Labels
- Improved Noise Schedule for Diffusion Training
- Self-Paced Learning for Latent Variable Models
- Part-dependent Label Noise: Towards Instance-dependent Label Noise
- Tri-net for Semi-Supervised Deep Learning