When clean training images are scarce, generative models can be trained on large piles of corrupted images plus a small clean set. The best quantitative result on this trade-off prices one clean ImageNet image at about 1.5 to 1.75 images corrupted with Gaussian noise. That price was measured with Fréchet Inception Distance (FID), a score that compares classifier features of generated and real images. FID is known to reward matching textures and to overlook broken structure such as fused fingers or distorted faces. This proposal asks whether the price of noisy data changes when quality is judged by people, or by measures that are sensitive to structure, instead of by FID. It also asks where the diversity of a mixed dataset has to live: in the small clean set or in the large noisy one.
The stakes are practical. Anyone deciding whether to buy, scrape, or clean data needs the right exchange rate. A rate that is accurate under FID and wrong under human judgment would lead to systematically bad data budgets. It would also give a concrete case of how the gap between objective and subjective quality propagates into data-scaling decisions.
Core Claim
How Much is a Noisy Image Worth? Data Scaling Laws for Ambient Diffusion trains diffusion models (generators that learn to turn random noise into images by repeated denoising) on mixtures of clean and noisy images. It uses a standard denoising loss on the clean images and a loss based on Tweedie's formula on the noisy ones. On ImageNet 64×64 at noise level σ = 0.2, the reported FIDs are 1.41 with all-clean data, 1.68 with 10% clean plus 90% noisy, 10.57 with the 10% clean images alone, and 5.32 with all-noisy data. The paper's mechanism is that clean images anchor fine high-frequency detail and noisy images mainly supply low-frequency layout.
This proposal starts from that mechanism. FID is biased toward texture, a bias documented by Human Preference Score, which observes that FID and IS favor anatomically broken images whose textures look natural. Defect studies such as AGIDefect-4K find that human quality ratings are dominated by small structural errors on faces and bodies. Putting these together, FID should mostly measure the part of quality that clean data provides and undercount the part that noisy data provides. Our central, falsifiable claim is that the clean-to-noisy exchange rate depends on the metric used to measure it.
To make this precise, for a quality measure $M$ and a model trained on $n_c$ clean and $n_n$ noisy images, define the exchange rate $\lambda_M$ as the number of noisy images that buy the same improvement in $M$ as one clean image:
$$ M(n_c, n_n) = M\!\left(n_c + n_n/\lambda_M,\ 0\right). $$
Here $M(a, b)$ is the quality of a model trained on $a$ clean and $b$ noisy images, interpolated along a measured clean-only scaling curve. We predict that $\lambda_{\text{human}} < \lambda_{\text{FID}}$ at moderate noise, meaning noisy images are worth more to human judges than FID implies. We further predict the gap is largest on face data, where structure dominates perception. If human-derived and FID-derived rates agree within their confidence intervals, that is also a useful result: FID would then be a safe instrument for data pricing despite its known failure modes.
The second claim concerns diversity. If clean data mainly supplies high-frequency detail, and detail statistics are shared across content, then a clean set drawn from a narrow slice of the distribution might suffice as long as the noisy set covers everything. The alternative is that clean coverage matters per slice, as the balanced-composition result in Gender imbalance in medical imaging datasets would suggest by analogy. That paper found that 50/50 balance beat larger imbalanced training sets for both groups. Which of the two holds tells practitioners whether to spend their cleaning budget on breadth or on volume.
Research Program
The work has three parts.
Part 1: reproduce and extend the FID scaling grid. On CIFAR-10 and CelebA-HQ we rerun the Ambient Diffusion training recipe over a grid of clean fractions and total sizes at Gaussian noise levels σ ∈ {0.05, 0.1, 0.2}. We also train clean-only models at several sizes to obtain the reference curve the exchange-rate definition needs. ImageNet 64×64 is too expensive for the full grid, so it receives a sparse set of points that includes the published 10%/90% configuration as a replication check.
Part 2: re-score every model with structure-sensitive and human measures. The human protocol borrows the two-alternative forced choice design behind the BAPPS judgments in The Unreasonable Effectiveness of Deep Features as a Perceptual Metric. Raters see one unconditional sample from each of two models and pick the more realistic or less defective image. Judgments are pooled into per-model Bradley–Terry scores, and $\lambda_{\text{human}}$ is computed on that scale. That paper reports that humans agree with each other 73.9% of the time on its patch task, which warns that individual judgments are noisy. We will therefore estimate inter-rater agreement and report bootstrap intervals on every exchange rate.
Alongside the human scores we compute three automated measures:
- the defect-detection score of the AGIDA model from AGIDefect-4K, which is weak (reported detection AUC 0.65) and so is treated only as a secondary signal;
- precision and recall in feature space, which separate fidelity from coverage;
- within-set pairwise LPIPS, a deep-feature image distance used here to track sample diversity, so that a model cannot win on quality by collapsing to a few modes.
Part 3: manipulate diversity explicitly. On CelebA-HQ we hold the clean budget fixed and draw it either from one attribute slice (for example, a single age-and-gender group with frontal pose) or evenly across slices. The noisy set covers all slices in both cases. Quality is then scored separately on slices that are absent from the clean set, giving an out-of-distribution test inside a single dataset. On ImageNet 64×64 the same manipulation uses class subsets.
As an exploratory extension toward the realistic corruptions that the Ambient paper names as uncharacterized, we add Gaussian blur and JPEG compression. The Tweedie loss does not apply directly to these corruptions. We therefore use corrupted images only at diffusion noise levels high enough to mask the corruption. This design choice is our own and could fail, and it is reported as such rather than as part of the core claim.
Baselines
Clean-only, the do-nothing reference. A model trained on only the clean subset (10.57 FID on ImageNet 64×64 at 10% clean in the Ambient paper), plus clean-only models at larger sizes that form the reference curve. Every method must beat the matching clean-only point under each metric before its exchange rate means anything.
Ambient Diffusion mixed training under FID. The strongest existing method and the object of study (Ambient Diffusion scaling laws). It reports 1.68 FID for 10% clean plus 90% σ = 0.2 noisy on ImageNet 64×64, against 1.41 all-clean, with an exchange rate of 1.5 to 1.75. We use its architecture, loss, and training length, and our replication must land near 1.68 before any new measurement is trusted. "Beating" it here does not mean a lower FID. It means showing that its FID-based price misforecasts human-judged quality by more than the bootstrap interval. If no such misforecast appears, the proposal's central claim fails.
Restore-then-train. Each noisy image is passed through a pretrained denoiser and the result is treated as clean, with the same model size, steps, and tuning budget as the mixed models. This is the obvious practitioner shortcut, and it ties to the misalignment literature. Restoration networks trained with pixel losses output the average of plausible details, the oversmoothing documented by Wei Liu et al. and Alexandre Alahi et al., so we expect it to look acceptable under FID and worse under human texture judgments.
Random clean subsets. This is the diversity ablation: clean subsets drawn at random rather than narrow or balanced by design, at equal size. It isolates whether composition, rather than count, drives the out-of-distribution slice scores.
Distribution-aware fine-tuning, a secondary check. We apply the FID-as-reward fine-tuning of Optimizing Visual Generative Models via Distribution-wise Rewards on top of the best mixed model, with its reported rolling reference-pool settings. That paper reports EDM2-S improving from 2.57 to 2.52 FID on ImageNet. The check asks whether optimizing FID after the fact widens or narrows the human-versus-FID gap; a widening gap would be direct evidence of metric exploitation.
Why not just…?
Why not just use better automated metrics and skip the humans? This is the most serious alternative. If precision/recall or a defect detector reproduces the human exchange rate, human studies are unnecessary for future pricing, and we would report that. Because the answer is unknown, the experiments rank every automated metric by how closely its $\lambda_M$ tracks $\lambda_{\text{human}}$, and identifying a cheap faithful proxy is a listed outcome. There is prior reason for doubt: zero-shot multimodal judges score defect AUCs of 0.29–0.38 on AGIDefect-4K and the trained AGIDA reaches only 0.65, so no existing automated judge is clearly reliable for structural errors.
Why not just filter out the noisy images, as preference-data work does? FiFA shows that discarding more than 99% of noisy human-feedback pairs can beat full data, with a 54.8% versus 37.0% human win rate. That noise, however, consists of contradictory labels, whereas ours is measurement corruption with a known form. The Ambient results already show that discarding the noisy images costs far more than it saves (10.57 versus 1.68 FID). The open question is how much the noisy data is worth, not whether to throw it away.
Why not just denoise first? This is run as a baseline above. If restore-then-train matches mixed training under human judgment at equal compute, the practical value of corruption-aware training shrinks, and we would say so.
Why not just assume FID is good enough because the 1.68 versus 1.41 gap is small? A small FID gap is exactly what a texture-biased metric would show if noisy data repaired structure that FID barely registers. Treating the gap as small assumes the conclusion, and the human-scored grid tests it directly.
Decisive Checks
The first check is replication. Our ImageNet 64×64 mixed model must land near the reported 1.68 FID, and our CelebA-HQ grid must recover an FID exchange rate of similar size. No human study on these models was found in the file ("no reported result found on Lacuna for human evaluation of Ambient Diffusion models"), so the human numbers will be new.
The main test compares $\lambda_{\text{human}}$ and $\lambda_{\text{FID}}$ with bootstrap intervals at each noise level and dataset. The claim is supported if the human rate is reliably lower on CelebA-HQ, and refuted if the intervals overlap everywhere.
The diversity test compares narrow and balanced clean sets of equal size on slices that are absent from the clean set. A narrow set that matches the balanced one supports the claim that "detail transfers." A large gap supports per-slice clean coverage, which would echo the medical-imaging balance finding in a generative setting.
Within-set LPIPS and recall are reported for every model so that no quality gain can hide a loss of diversity.
References
- How Much is a Noisy Image Worth? Data Scaling Laws for Ambient Diffusion.
- Human Preference Score: Better Aligning Text-to-image Models with Human Preference
- AGIDefect-4K: A Richly Annotated Dataset for AI-Generated Image Defect Detection, Localization and Explanation
- Gender imbalance in medical imaging datasets produces biased classifiers for computer-aided diagnosis
- The Unreasonable Effectiveness of Deep Features as a Perceptual Metric
- Bayesian Tensor Inference for Sketch-Based Facial Photo Hallucination
- Perceptual Losses for Real-Time Style Transfer and Super-Resolution
- Optimizing Visual Generative Models via Distribution-wise Rewards
- Automated Filtering of Human Feedback Data for Aligning Text-to-Image Diffusion Models