Image and video generators sample a new face every time they start from new random noise. A diffusion model (a generator that turns random noise into an image by removing noise step by step) prompted twice with "a wizard" will usually draw two different wizards. Current fixes attach an identity anchor, such as a learned token or attention borrowed from a reference image, and push toward it with a fixed strength. That fixed strength forces a trade-off. Push hard and the character is locked, but it can no longer change pose, move into a new scene, or share a frame with other characters. Push softly and the identity drifts. Benchmarks show how far drift still goes. Long-video models reach only 41.3% PrecisionID on MovieBench, and character consistency on LoCoT2V-Bench falls to 30–55% while background stability stays above 90%.
We propose to control identity strength the way hybrid thermal systems control temperature. A weak passive anchor handles the bulk of the load. An active feedback loop measures identity drift during denoising and adds identity guidance only when, and only as much as, the measurement shows drift. In two sentences: keep a light identity anchor that never over-constrains the image, and at each denoising step check whether the character in the model's current best guess still matches the reference, adding correction only when it does not. The claim is that this gives a better identity-versus-prompt trade-off than any fixed or scheduled guidance strength, and that the gain is largest where fixed strength fails most: large pose and viewpoint changes, and scenes with several characters.
Core Claim
The mechanism comes from hybrid thermal management. Xu et al. combine passive graded insulation with feedback (PID) heating of a mounting plate and hold temperature to better than 0.05°C. Their sensitivity analysis shows that the feedback loop cancels disturbances the passive structure alone cannot, such as changes in pad conductivity. Wu et al. show the complementary point in battery cooling. An active liquid loop continuously drains a phase-change buffer, a material that soaks up heat as it melts. Draining it before it fully melts keeps the pack under 309.8 K and lets up to 90% of the buffer's heat capacity be used.
The generative analogue of that saturated buffer is the over-strong anchor. Shared attention or a heavily tuned token that copies too much of the reference produces the attribute leakage and pose rigidity reported for StyleAligned-style sharing and for OneActor.
Feedback Guidance has already shown that closed-loop control helps diffusion. Scaling guidance by an estimate of how far the sample has drifted from the text condition cut ImageNet 512 FID from 5.00 to 3.76. That method senses drift with the model's own conditional and unconditional predictions under an additive-error assumption, and it targets text adherence, not identity. We differ in three ways:
- We sense identity with an external measurement on the predicted clean image.
- We measure each character separately, inside its own image region.
- We actuate an existing identity anchor rather than text guidance.
The per-character measurement is what lets the method address multi-subject interference, which a single global scale cannot.
Let $\hat{x}_0^{(t)}$ be the model's predicted clean image at step $t$. Let $s_k(\hat{x}_0^{(t)})$ be the DINOv2 cosine similarity between character $k$'s masked region and that character's reference, and let $\tau_k$ be a target similarity. The drift error is $e_k^{(t)} = \max(0, \tau_k - s_k(\hat{x}_0^{(t)}))$. The controller sets a per-character guidance weight
$$ w_k^{(t)} = \mathrm{clip}\Big(K_p\, e_k^{(t)} + K_i \sum_{t' \ge t} e_k^{(t')},\; 0,\; w_{\max}\Big), $$
where $K_p$ and $K_i$ are the proportional and accumulated-error gains and $w_{\max}$ caps the correction. The denoising prediction becomes
$$ \epsilon = \epsilon_{\text{text}} + \sum_k w_k^{(t)}\, m_k \odot \big(\epsilon_{\text{anchor},k} - \epsilon_{\text{text}}\big). $$
Here $\epsilon_{\text{text}}$ is the prediction with the ordinary prompt, $\epsilon_{\text{anchor},k}$ is the prediction with character $k$'s identity anchor active, and $m_k$ is that character's spatial mask. The anchor itself is a cheap passive element: a OneActor-style projector or a light textual-inversion token, run at low strength.
Because the actuator is a difference between two predictions, no gradients flow through the identity measurement. The extra cost is one additional forward pass per character plus a decode and DINOv2 call every few steps.
Research Program
We build the controller on top of three passive anchors: a OneActor projector, a The Chosen One token plus LoRA (small low-rank weight updates), and reference-attention sharing. For video, we add the temporally correlated noise prior of Magic-Me as a further passive stabilizer. Masks $m_k$ come from cross-attention maps of each anchor token and are refreshed as the image forms.
The program has three analyses. The first measures when during denoising identity drift arises. We expect most error to accumulate in a middle band of steps, which would explain why a fixed schedule is a poor fit. The second asks whether weakening the passive anchor while adding feedback reduces leakage without losing identity. This is the analogue of draining the buffer before it saturates. The third checks controller stability: whether proportional gain alone oscillates, meaning the face alternates between over- and under-corrected across steps, and whether the accumulated-error term removes residual offset.
Coating durability supplies the evaluation method rather than the mechanism. Microstructured coatings trade function for fragility: taller features perform better but snap under sideways load. Coating studies also report wear in incompatible units, 100 rubbing cycles in Chen et al. against 1600 in Liu et al., which blocks cross-comparison. Identity methods share both problems. A stronger lock is more brittle to prompt change, and consistency is reported on unaligned prompt sets.
We therefore define a graded "prompt load" test. Load levels step up through four stages: attribute edits, then pose and viewpoint change, then scene and prompt length, then the number of co-present characters from one to six. We report each method's identity-retention curve against load and the critical load at which retention falls below a fixed threshold. This plays the role that the collapse of a water-repellent surface into the fully wetted state plays in coating tests: a single failure point that can be compared across methods. We use the load levels only as a benchmarking device and make no physical claim from the analogy.
Baselines
The naive reference is the plain text-conditioned model with no anchor. It fixes the floor for identity and the ceiling for prompt adherence.
The strongest image-side competitor is OneActor with its constant cluster guidance and semantic-interpolation sweep. It already traces an identity-versus-prompt Pareto front, meaning the set of best achievable trade-off points, at a tuning cost of 3–6 minutes. The file reports no identity-score numbers for OneActor on a standard benchmark, so no reported result was found on Lacuna for that comparison, and we will rerun it. Anchor, backbone, prompts and seeds are held identical between OneActor and our controller, and both receive the same budget of hyperparameter trials on a held-out prompt split. Beating OneActor means our Pareto front dominates its front at a matched number of network forward passes.
The Chosen One is a second anchor baseline. It takes about 20 minutes of extraction on an A100 and is run with its released procedure.
On video, the strongest reported numbers are Magic-Me's on CustomConcept101 prompts: DINO identity similarity 0.585, CLIP-T text alignment 0.315 and CLIP-I temporal consistency 0.835. Beating it means raising DINO identity above 0.585 without CLIP-T falling below 0.315 or temporal consistency below 0.835.
For long-form stories, the reference points are the 41.3% PrecisionID on MovieBench and LongLive's 70.56% overall on LoCoT2V-Bench. The comparison uses the character-consistency sub-score of LoCoT2V-Bench, where reported models sit at 30–55%.
Two ablations isolate the key ingredient, which is feedback from an identity measurement. The first replaces the controller with an open-loop schedule fitted to the average drift profile, so it has the same average strength over time but no per-sample sensing. The second applies the Feedback Guidance estimator unchanged to the anchor condition, so it senses drift with the model's own predictions instead of an identity measurement and uses one global weight rather than per-character weights.
Why not just…?
Why not just tune a constant identity guidance scale? This is the real threat. OneActor's interpolation already moves along the trade-off curve, and a well-chosen constant may be close to optimal for single-character prompts at low load. Our hypothesis is that one constant cannot fit all prompts: the right strength differs between a frontal portrait and a back view in a crowd. We settle this by overlaying the full front of constant scales against ours at each load level. If the fronts coincide for one character but separate at high pose and multi-character load, the claim narrows to that regime. If they coincide everywhere, the idea is wrong.
Why not just use a fixed time schedule? A schedule captures average drift timing at no sensing cost. The fitted-schedule ablation tests exactly this. Feedback earns its cost only if per-sample variation in drift is large, which the first analysis measures directly.
Why not just draw N samples and keep the best by identity score? Best-of-N selection with the same DINOv2 scorer is simple and strong. We give it the same number of forward passes as the controller. We expect it to fail with several characters, because the probability that all of them are correct in one sample drops roughly multiplicatively with each character added. It is also useless for video frames, which must share one trajectory. We report the comparison at one character, where selection may win.
Why not just use Feedback Guidance on the identity condition? Its drift estimate uses the model's own predictions and cannot tell which of several characters is drifting. The second ablation shows whether an external, per-region identity measurement matters.
Proposed Experiments
Image experiments use DreamBench++, which has 150 references and 1,350 prompts and on which human identity preservation is the hardest category. We re-bin its prompts into our load levels and add composed scenes with two to six characters, directly probing OneActor's reported failure beyond four subjects.
Video experiments follow the Magic-Me protocol on CustomConcept101 prompts. Story-level experiments use MovieBench and LoCoT2V-Bench.
Scoring guards against circularity. The controller senses identity with DINOv2, so evaluation uses held-out identity measures: the benchmarks' own consistency scores, a separate face-identity model not used in the loop, and a paired human preference study. Text alignment, leakage of reference attributes, and wall-clock cost are reported alongside identity.
Three results decide the question, and each has a pre-stated failure outcome:
- Pareto dominance. The controller's front must dominate the tuned constant-scale front at matched compute on the high-load bins. Failure outcome: the fronts coincide at every load level.
- Weak anchor plus feedback. Pairing a weakened anchor with feedback must reduce measured attribute leakage relative to a strong anchor at equal identity. Failure outcome: leakage is no lower at equal identity.
- Critical load in multi-character scenes. The critical load, the number of co-present characters before identity collapses, must rise over OneActor and best-of-N. Failure outcome: it stays at OneActor's four-subject limit or at best-of-N's level.
References
- MovieBench: A Hierarchical Movie Level Dataset for Long Video Generation
- LoCoT2V-Bench: A Benchmark for Long-Form and Complex Text-to-Video Generation
- High-precision, Temperature Control Based on Grading-structure and PID-feedback Strategies
- Research on Performance Optimization of Liquid Cooling and Composite Phase Change Material Coupling Cooling Thermal Management System for Vehicle Power Battery
- Style Aligned Image Generation via Shared Attention
- OneActor: Consistent Subject Generation via Cluster-Conditioned Guidance
- Feedback Guidance of Diffusion Models
- The Chosen One: Consistent Characters in Text-to-Image Diffusion Models
- Magic-Me: Identity-Specific Video Customized Diffusion
- Facile design of PTFE-kaolin-based ternary nanocomposite as a hydrophobic and high corrosion-barrier coating
- Fluorine-free and durable hydrophobic coating made with corn straw-derived silane modified epoxy resin
- DreamBench++: A Human-Aligned Benchmark for Personalized Image Generation