Conditional image generators make outputs match their prompt using classifier-free guidance (CFG). CFG runs the network twice at every denoising step, once with the prompt and once without, and pushes the output along the difference between the two predictions by a fixed factor $w$. This doubles inference cost. Guidance distillation removes that cost by training a single network to output the guided result in one pass. Every existing distillation method, though, copies CFG itself, and CFG is a heuristic. It is not the update that would sample the distribution we actually want, which is the model's output reweighted toward images the class strongly predicts. Stochastic optimal control frames guidance as steering a noisy process at minimum cost, and it gives the correct update: the gradient of a value function, meaning the expected future reward from the current noisy state. The obstacle is that value functions are notoriously hard to learn in high dimensions (the "curse of dimensionality"). The most careful work in this direction, Adaptive Diffusion Guidance via Stochastic Optimal Control, stopped at 2D toys.
We propose to learn this guidance value function on top of a frozen multi-step model, using short-horizon value regression that is known to scale to hundreds of dimensions. We then distill the resulting optimally guided velocity into a one-step student. The student needs neither the second network pass nor runtime differentiation of the value function. In two sentences: replace "distill CFG" with "distill the exact reward-tilted control," and make that target computable in high dimensions by borrowing value-learning machinery from optimal control. A controlled dimension sweep is built in, because theory suggests CFG's errors may fade as dimension grows, and if they do, the method should not be used.
Core Claim
Consider the reverse diffusion process of a pretrained model, which runs from noise at $t=1$ to data at $t=0$. Add a control $u$ and penalize it by its KL cost relative to the unguided process. Reward the endpoint with $\gamma \log p(c\mid x_0)$, where $c$ is the class label, $\gamma$ trades prompt adherence against diversity, and $p(c\mid x_0)$ is a classifier. The optimal guided process then samples the tilted distribution $p(x)\,p(c\mid x)^\gamma$. Because this problem has quadratic control cost and control-affine dynamics, a logarithmic change of variables makes the value equation linear. The optimal control is therefore a gradient of a conditional expectation:
$$u^*(x_t,t) = \sigma_t^2\,\nabla_x \log \mathbb{E}\big[\,p(c\mid x_0)^{\gamma}\,\big|\,x_t\big],$$
Here the expectation runs over the unguided teacher's reverse process started from $x_t$, and $\sigma_t$ is its noise scale. CFG substitutes $p_t(c\mid x_t)^\gamma$ for the expectation. Those two quantities coincide only at $t=0$. That substitution causes the effects reported for CFG: mean overshoot, variance shrinkage, and off-manifold artifacts at large $w$ (Classifier-Free Guidance: From High-Dimensional Analysis to Generalized Guidance Forms, Manifold-Optimal Guidance).
The claim has three parts. First, a scalar value network $V_\theta(x,t)$ approximating $-\log \mathbb{E}[p(c\mid x_0)^\gamma\mid x_t]$ can be learned in latent-image dimension by temporal-difference regression over short teacher branches. TD regression fits each state's value to a short simulated look-ahead plus the network's own estimate at the end of that look-ahead. Second, a one-step student trained on the target "teacher velocity plus $u^*$" gives better quality–diversity trade-offs than a student distilled from CFG at equal compute. Third, conditioning the student on $\gamma$ restores the runtime adjustability that fixed-$w$ distillation gives up.
Relation to Prior Work
Several papers motivate the gap. Adaptive Diffusion Guidance via Stochastic Optimal Control proves that optimal guidance strength is state-dependent and satisfies a Hamilton-Jacobi-Bellman equation, the PDE that defines a value function. On the 2D Cup task it raises reward from 5.191 to 8.099 and lowers KL from 1.657 to 0.766 against the best constant weight. It optimizes a scalar weight $w(x,t)$ for multi-step sampling only, it stays in 2D, and it names inaccurate scores in low-density regions as the barrier to scaling. We differ in three ways: a full-dimensional control rather than a scalar, a scalable value learner, and a distilled single-pass student. Manifold-Optimal Guidance also treats guidance as control, using a curved metric that penalizes off-manifold drift. It is training-free and keeps both network passes at every step, whereas we remove them.
Other papers supply the method. Path Integral Value Matching learns value functions for linearly solvable control problems by TD regression over 8-step branches and reaches 200 dimensions, where SOCM runs out of memory at 80 and Adjoint Matching diverges at 200. That paper explicitly leaves open distilling $-\sigma^\top\nabla V$ into a direct policy to avoid runtime differentiation, and our second stage answers that question in a generative setting. MeanFlow and SoFlow supply the one-step student objectives. Both already fold a fixed-$w$ CFG velocity into training, and we swap that velocity for teacher-plus-$u^*$.
Other work constrains the design. Classifier-Free Guidance: From High-Dimensional Analysis to Generalized Guidance Forms shows that CFG's distortions in Gaussian mixtures shrink as $O(1/\sqrt{d})$ and that guidance matters mainly in an early class-selection phase. That result directly threatens our claim, and we test it rather than assume it away. Cideron et al. document diversity collapse when guidance is distilled and patch it with diversity rewards. Our hypothesis is that much of that collapse comes from the CFG target itself. Designing RL for Diffusion Models: A Unified Path-Space View shows that path-space objectives become unstable near $t\to 0$ without bounded weighting, and we adopt its scale-bounded weighting in the TD loss.
Research Program
Stage one learns the value function. Starting from the frozen teacher, we sample noisy states $x_t$, roll the unguided teacher sampler forward over $k$ steps, and regress $V_\theta(x_t,t)$ onto the log-sum-exp of $V_\theta$ at the branch endpoints. These are short branches in the style of PI-VM, rather than full trajectories. At $t=0$ the target is $-\gamma\log p(c\mid x_0)$. Because the branches are short, memory does not grow with the trajectory length. We expect the curse of dimensionality to be manageable for a structural reason. The terminal reward depends on $x_0$ mainly through a few class-discriminative directions, so we parameterize $V_\theta$ as a small head over the teacher's own intermediate features plus the CFG proxy $\log p_t(c\mid x_t)$. In that form the network learns a correction to CFG rather than a value function from scratch. The branch samples are drawn from the unguided teacher, which is the reference process in the KL cost. This keeps the variance blow-up that SOCM suffers when the initial policy is far from optimal out of play.
Stage two distills. The target velocity is $v_{\text{teacher}}(x_t,t) + u^*$, where $u^*$ comes from automatic differentiation of $V_\theta$, done only offline at training time. We train a $\gamma$-conditioned one-step student with the SoFlow objective, chosen for its lower memory than MeanFlow (35.4 GB against 51.4 GB). Stage three is the analysis: we quantify how the gap between the $u^*$ target and the CFG target scales with dimension and with diffusion time.
Baselines
The naive reference points come first. One is the one-step student trained with no guidance at all. The other is the multi-step teacher with standard CFG, which bounds achievable quality; MeanFlow reports 250-step diffusion at FID 2.0–2.3 on ImageNet 256.
The strongest competitor is fixed-$w$ guidance internalized into a one-step model. SoFlow-XL/2 reports 1-NFE FID-50K of 2.96 on ImageNet 256 (SoFlow), and MeanFlow reports 3.43. It is the strongest because it already reaches near-teacher quality in one pass. If CFG's errors truly vanish in high dimension, nothing can beat it. We will retrain it ourselves on the same backbone, data, epoch budget, and teacher, and sweep $w$ with the same tuning budget we give $\gamma$. Beating it means a better precision–recall frontier, with higher recall at matched FID or lower FID at matched recall, plus a single $\gamma$-conditioned model that matches a family of separately trained fixed-$w$ students.
A second baseline distills time-scheduled and power-law CFG into the same student. Power-law CFG raised EDM2-S recall from 0.582 to 0.631 in multi-step sampling (Classifier-Free Guidance: From High-Dimensional Analysis…). This tests whether time dependence alone captures the benefit.
A third baseline is Manifold-Optimal Guidance as a multi-step, two-pass reference; it reports FID improving from 19.14 to 8.78 on DiT-XL/2.
Three ablations isolate the key ingredient. The first replaces $V_\theta$ with the CFG proxy and keeps everything else fixed. The second learns the same value function with full-trajectory SOCM or adjoint matching instead of short branches, to measure memory use and stability. The third evaluates the teacher with $\nabla V_\theta$ computed at runtime and no distillation, to measure how much distillation itself costs.
Why not just…?
Why not just distill CFG with a well-tuned or scheduled weight? This is the real threat. The $O(1/\sqrt{d})$ result implies the exact target and the CFG target may converge in high dimension. Against that, the same analysis places guidance's effect in the early class-selection phase, which is exactly where the state-dependent gains of Adaptive Diffusion Guidance appear. The Gaussian-mixture dimension sweep settles the question. If the gap closes as $1/\sqrt{d}$, we report that and recommend scheduled CFG distillation instead.
Why not just fine-tune the one-step student with reward and KL, as in Diff-Instruct* or path-space RL? This optimizes the same tilted objective directly (David and Goliath, Path-Space View). A one-step generator has no path, however, so the KL term must be replaced by a surrogate divergence. RL objectives also carry the variance problems the path-space paper documents. Our approach turns the problem into plain regression. We run DI*-style post-training on the same student and budget as a baseline, and we do not presume the outcome.
Why not just learn the guidance policy directly with SOCM or the adjoint method? On Quadratic OU, SOCM runs out of memory at 80 dimensions and Adjoint Matching diverges at 200, while short-branch value matching scales to 200 at 10–20x the speed (PI-VM). Our second ablation re-measures this in the guidance setting.
Why not just use MOG at inference? It keeps two passes over many steps and cannot produce a one-pass sampler. We include it only as a quality reference.
Proposed Experiments
The decisive experiment is a Gaussian-mixture study. There the tilted score, and hence $u^*$, is available in closed form, so we can sweep $d$ from 2 to 256 in the dimensions used by SOCM and PI-VM. We measure the $L^2$ error of the student's control against the analytic $u^*$, the KL to the tilted target, and the error in mode weights. We do this separately for the CFG-distilled student and the $u^*$-distilled student, and plot the gap against $d$ and $t$. We also reproduce the Cup benchmark, which checks that a scalar-$w$ restriction of our method recovers the reported 8.099 reward. A distilled student with $\gamma$ conditioning should approach that number in one pass.
At scale, we use ImageNet 256 in latent space with a SoFlow-XL/2 student distilled from a DiT teacher, a pretrained latent classifier as the reward, and $\gamma$ swept. We report FID-50K, precision and recall, classifier log-likelihood, NFE, wall-clock latency, and the memory and time of value training. CIFAR-10 serves as a mid-dimension check point.
The proposal is falsified under either of two outcomes. If the sweep shows the target gap decaying as $1/\sqrt{d}$ with no early-time residue, the premise fails. It also fails if the CFG-proxy ablation matches the full method on ImageNet recall at equal FID. Either outcome would itself be a useful negative result: it would show that, for guidance distillation, the dimensionality of image models works in our favor rather than against us.
References
- Adaptive Diffusion Guidance via Stochastic Optimal Control
- Understanding Classifier-Free Guidance: High-Dimensional Theory and Non-Linear Generalizations
- Manifold-Optimal Guidance: A Unified Riemannian Control View of Diffusion Guidance
- Path Integral Value Matching for Linear Quadratic Stochastic Optimal Control
- Mean Flows for One-step Generative Modeling
- SoFlow: Solution Flow Models for One-Step Generative Modeling
- Diversity-Rewarded CFG Distillation
- Designing Reinforcement Learning for Diffusion Models: A Unified Path-Space View
- David and Goliath: Small One-step Model Beats Large Diffusion with Score Post-training