Many image generators now produce a sample by integrating an ordinary differential equation (ODE), a deterministic rule that moves a point from random noise to an image along a learned velocity field. Probability-flow diffusion samplers and flow matching both work this way. Separately, concept-bottleneck generative models route generation through a small set of human-named attributes, such as "smiling" or "wing color", so that a user can flip an attribute and see the image change. Combining the two raises a measurable puzzle. Hard-bottleneck work reports that iterative diffusion is unusually resistant to concept edits compared with GANs, and gives no explanation (A Probabilistic Hard Concept Bottleneck for Steerable Generative Models). Our claim is that one property of the velocity field explains this resistance and also governs how robust the sampler is to its own errors. That property is how strongly the field pulls neighboring trajectories together, which control theory calls contraction. A field that contracts strongly forgets solver error and score error, which gives good few-step sample quality. The same field also forgets concept edits. The accuracy-interpretability trade-off in this setting is therefore a geometric trade-off about which directions the flow is allowed to forget.
The proposal has two halves. The first turns this observation into a quantitative, testable prediction of edit persistence. The second uses it to build a regularizer that contracts only in non-concept directions. If the idea is right, a concept-conditioned flow model can keep the error-robustness of contractive sampling without paying for it in steerability. If the prediction fails, we learn that edit resistance comes from something other than flow geometry, most likely leakage. That would itself point the field back toward bottleneck design.
Core Claim
Let $x_s$ be the sampler state at time $s\in[0,1]$, $v_s(x,c)$ the learned velocity field conditioned on a concept vector $c$, and $J_s = \partial v_s/\partial x$ its Jacobian along the trajectory. A small perturbation $e$ of the state injected at time $t$ evolves by the linearized ODE $\dot e = J_s e$. By standard contraction arguments, its size at the end is controlled by the logarithmic norm $\mu(J_s)$, which is the largest eigenvalue of the symmetric part of $J_s$:
$$ \|e(1)\| \le \exp\!\Big(\int_t^1 \mu(J_s)\,ds\Big)\,\|e(t)\|. $$
The same factor multiplies local discretization and score-estimation errors in finite-time ODE analyses. Strongly negative $\mu$ is therefore what makes deterministic sampling forgiving, and the contraction penalty in Contractive Diffusion Policies exploits exactly this.
A concept edit has two parts. It changes the conditioning input $c$, and it changes the state, either because an earlier part of the trajectory was generated under the old $c$ or because the user edits an intermediate representation. The final change is a sum of source terms $\partial v_s/\partial c\,\Delta c$, each propagated forward through the same linearized flow. The edit therefore survives only if the flow is not contracting along the directions the edit occupies.
We define a directional quantity. Let $u_s$ be the unit direction of $\partial x_s/\partial c_k$, the sensitivity of the trajectory to concept $k$, computed by forward-mode differentiation through the solver. The concept log-rate is then $\mu_k(s) = u_s^\top J_s u_s$. Our prediction is that the probability an edit of concept $k$ at time $t$ is realized in the final image is a monotone function of $\int_t^1 \mu_k(s)\,ds$. We also predict that uniform contraction lowers this quantity and lowers few-step error together. The remedy follows from the decomposition: penalize the log-norm of $J_s$ restricted to the complement of the concept directions, and hold $\mu_k$ near zero.
This differs from the closest existing work in specific ways. Concept Bottleneck Generative Models inserts a concept layer, plus an orthogonal "unknown concept" residual, into GANs, VAEs and diffusion models. It treats sampling as a heuristic and never asks how the bottleneck interacts with solver dynamics. scCBGM pairs flow matching with a concept bottleneck for single-cell editing. It uses a cross-covariance penalty against leakage and feeds concepts into every decoder layer to make interventions persist. That design choice is an empirical answer to the same edit-persistence problem, but the paper offers no account of why persistence fails or how it trades off against integration error. Contractive Diffusion Policies regularizes contraction isotropically, in every direction at once, for robot control, where nobody wants to steer the trajectory mid-flight. Our proposal is the first to make contraction direction-dependent and tied to named concepts. It also supplies a falsifiable account of the edit-resistance gap.
Research Program
The first component is analysis in a setting where everything is computable. We use Gaussian-mixture targets with a known ground-truth flow, the testbed used by Convergence Analysis of Probability Flow ODE for Score-Based Generative Models and Finite-Time Convergence Analysis of ODE-based Generative Models for Stochastic Interpolants, up to 128 dimensions. Mixture-component labels serve as concepts.
Here we can compute $J_s$ exactly and prove a two-sided statement about the linearized flow. Final edit magnitude is bounded below by a factor $\exp(\int \lambda_{\min,k})$ and above by $\exp(\int \mu_k)$. The terminal error bound improves with contraction only in the complement directions. We expect a clean result only for the linearized flow plus Lipschitz remainders. We will not claim tight non-asymptotic total-variation constants, since the file already notes that existing dimension bounds are conservative relative to observed linear scaling.
The second component is the architecture. We train concept-conditioned flow-matching image models on CelebA-HQ and CUB-200-2011 in which the concept vector enters the velocity network as a constant input along the trajectory. This placement matters for a gap the file flags: discrete hard bottlenecks seemed to clash with ODE smoothness (A Probabilistic Hard Concept Bottleneck for Steerable Generative Models). If binary concepts are inputs rather than part of the state, the field stays Lipschitz in $x$ for each fixed $c$. The binary jump then happens between ODEs rather than inside one, so we can reuse the variational hard bottleneck's leakage resistance without creating solver singularities. We will test this claim rather than assume it, by checking step-size error slopes on the trained model.
The third component is the regularizer. Following the power-iteration estimator from Contractive Diffusion Policies, we estimate the top eigenvalue of the symmetric part of $(I-P_s)J_s(I-P_s)$ and penalize it when it exceeds $-\alpha$. Here $P_s$ is the projector onto the span of the concept sensitivities $\partial x_s/\partial c_k$, obtained with one Jacobian-vector product per concept. A second term penalizes $|\mu_k(s)|$. All quantities use Jacobian-vector products, so for ten to twenty concepts the cost is a modest constant multiple of training. We will report the measured overhead.
Baselines
The naive reference point is an unconstrained flow-matching model with no concept bottleneck. Concepts are probed only by an external attribute classifier, and samples use Heun's second-order solver, which Finite-Time Convergence Analysis of ODE-based Generative Models for Stochastic Interpolants shows reduces iteration complexity relative to Euler. This model sets the fidelity ceiling at each step budget and has no steering at all. It tells us how much quality any interpretable model gives up.
The strongest existing interpretable competitors are the concept-bottleneck generative model of Concept Bottleneck Generative Models, run from the official prescient-design/CBGM code, and the variational hard bottleneck of A Probabilistic Hard Concept Bottleneck for Steerable Generative Models on DDPM.
The reported reference numbers are thin. On CelebA, the concept-bottleneck GAN reaches 25.6% steerability accuracy against 8.7% for a conditional GAN, as reported in Concept Bottleneck Generative Models. The hard-bottleneck paper reports a 46% average relative improvement in single-concept intervention accuracy over CB-AE. The file gives no absolute steerability or FID figure for it, so there is no reported absolute result on Lacuna for the hard bottleneck on CelebA-HQ or CUB. We will therefore rerun both methods ourselves rather than compare against quoted numbers. For few-step fidelity on CIFAR-10 the file does give a reference: Interpreting and Improving Diffusion Models from an Optimization Perspective reports FID 3.79 at 10 steps for its gradient-estimation sampler versus 16.86 for DDIM.
The ablation that isolates our key ingredient is the same concept-conditioned flow model under three conditions: no contraction penalty, isotropic contraction as in Contractive Diffusion Policies, and our concept-anisotropic contraction. Each is swept over penalty strength.
Fairness rules are the same throughout. All models use the same data splits, backbone size, and training compute. Penalty weights get equal tuning budgets, chosen on a held-out validation set by one scalar criterion: steerability at a fixed FID tolerance. Steerability for every method is scored by the same frozen attribute classifiers trained on real images.
Our method beats a baseline if it does both of the following. First, it dominates the isotropic-contraction curve in the plane of FID at 10 steps versus steerability. Second, at matched FID it exceeds the rerun hard-bottleneck and concept-bottleneck models' single-concept steerability, with confidence intervals across three seeds that do not overlap.
Why not just…?
Why not just feed the concept into every layer, as scCBGM does, and skip the geometry? This is the most serious alternative, and it may already capture much of the benefit. scCBGM reports that decoder skip connections make interventions persist on single-cell data. Our account predicts something specific, though. Even with concepts injected at every layer, the contribution of concept changes made early in the trajectory is still damped by the integrated log-norm, so a strongly contracting skip-connected model should realize edits only through late, low-level changes. Our skip-connected baseline already includes per-layer injection. If that model shows no dependence of edit success on $\int\mu_k$, our mechanism is unnecessary and we will say so.
Why not just raise the guidance scale until the edit takes? Stronger classifier-free guidance can overpower contraction. However, Classifier-Free Guidance is a Predictor-Corrector shows guidance does not sample the intended sharpened distribution and behaves differently under deterministic and stochastic solvers. On ImageNet, a corrected predictor-corrector variant lowers FID from 3.90 to 3.35 at scale 1.1, which suggests plain guidance leaves quality on the table. We will sweep guidance scale on the unregularized model and plot it in the same FID-steerability plane. If guidance alone reaches our curve, the regularizer adds nothing.
Why not just invert the image to noise and regenerate with the new concept? A deterministic ODE gives an invertible map, so one can recover a sample's starting noise and re-run the flow under an edited $c$. This bypasses mid-trajectory resistance entirely. It costs a full extra integration, and it does not keep unedited attributes fixed unless the flow's sensitivity to $c$ is well separated from the residual directions, which is exactly what our regularizer encourages. We will include inversion-then-regenerate as a baseline for both steerability and preservation of non-target concepts.
Why not just explain an unconstrained model post hoc? Post-hoc saliency does not let a user change the output. Counterfactual Concept Bottleneck Models shows that pixel-level counterfactuals are noisy and hard to read. The question here is intervention, not attribution, so post-hoc methods do not compete on the metric that matters.
Proposed Experiments
The first decisive check is the predictive test on CelebA-HQ and CUB. We take many trained models across contraction strengths, inject single-concept flips at times $t\in\{0.1,\dots,0.9\}$, and measure, per concept and per injection time, whether the frozen classifier registers the flip. We then regress success on the measured $\int_t^1\mu_k(s)\,ds$. The mechanism is supported if this one scalar explains most of the variance in edit success across models, concepts and times. It is refuted if success tracks leakage instead, which we measure by how well residual features predict concepts, as in the hard-bottleneck paper, rather than tracking flow geometry.
The second check is the trade-off curve. We plot FID at 5, 10 and 50 Heun steps against steerability for the unregularized, isotropic and anisotropic models. The prediction is that isotropic contraction moves models along a downward-sloping frontier, improving FID while losing steerability, and that anisotropic contraction moves the frontier outward. We also report preservation of non-target concepts, because an edit that changes everything is not interpretable.
The third check is the ground-truth testbed. On Gaussian mixtures up to 128 dimensions, the bounds can be compared with exact flows. We verify that final error slopes stay at the solver's order when a binary concept is supplied as a conditioning input, which addresses the discreteness concern. We also verify that terminal error tracks the complement-direction log-norm rather than the full one.
A fourth, smaller experiment asks whether the separation helps memorization. Implicit Regularisation in Diffusion Models shows that coarse discretization and SGD noise prevent memorization. We test whether anisotropic contraction changes nearest-training-neighbor distances on CIFAR-10 relative to isotropic contraction. This is a secondary check, not a headline claim.
References
- A Probabilistic Hard Concept Bottleneck for Steerable Generative Models
- Contractive Diffusion Policies
- Concept Bottleneck Generative Models
- scCBGM: Single-Cell Editing via Concept Bottlenecks
- Convergence Analysis of Probability Flow ODE for Score-Based Generative Models
- Finite-Time Convergence Analysis of ODE-based Generative Models for Stochastic Interpolants
- Interpreting and Improving Diffusion Models from an Optimization Perspective
- Classifier-Free Guidance is a Predictor-Corrector
- Counterfactual Concept Bottleneck Models
- Implicit Regularisation in Diffusion Models: An Algorithm-Dependent Generalisation Analysis