Annealed Identity Locks: Borrowing Polymer Processing Rules for Consistent Characters in Diffusion Models

Image generators that denoise random noise into pictures, known as diffusion models, cannot keep a character looking the same across a story. Ask twice for "a wizard cooking" and "a wizard reading" and you get two different wizards. Current fixes bind the images together too tightly or too loosely. Fine-tuning on the character or sharing attention across all images makes the face stable, but accessories and backgrounds get stuck to the character and new poses become hard to produce. Lighter methods keep the prompt editable but let the face drift. Organic electronics has a close analogue. A conjugated polymer needs a flat, tightly stacked backbone to conduct charge, but it needs floppy side chains to dissolve so it can be printed. Chemists found a way around this, and we propose to copy the rule rather than the metaphor. Their rule is to use weak, local, reversible "locks" that let go when the material is hot and fluid and snap shut as it cools slowly through ordered intermediate stages. Cooling too fast traps disorder.

We treat the noise level during denoising as temperature. Our method is training-free. It couples images of the same character only through matched subject regions, and a measured order parameter sets the strength of that coupling at every denoising step. The order parameter is how much the frames already agree about the subject. At high noise, coupling stays off so layout and pose are free. As agreement emerges, coupling tightens. If agreement collapses, for instance because a prompt demands a drastic pose change, coupling releases again. The testable claim is that this feedback-controlled, reversible schedule beats any fixed "share attention from step $a$ to step $b$" schedule, because the right moment to lock differs from prompt to prompt. A fixed schedule is the equivalent of quenching a polymer film at a fixed rate.

What the polymer literature actually supplies

Two results from organic electronics shape the design, and they play different roles.

The first is a negative result that constrains the design. Trade-off between processability and device performance in donor–acceptor semiconductors revealed using discrete siloxane side chains built an exact library of siloxane side chains. Longer and branched chains made the polymer more soluble but twisted the backbone and broke long-range order. Hole mobility fell from 0.18 to 0.01 cm²V⁻¹s⁻¹ between the Si₇ and Si₁₅ chains. The imide-polymer review Progress in the synthesis of imide-based N-type polymer semiconductor materials attributes the same kind of drop to wider spacing between chains. The side chains in these polymers act everywhere and permanently. The diffusion analogues are LoRA-style fine-tuning, which bakes a fixed change into the weights, and batch-wide attention sharing. We therefore avoid global, always-on coupling.

The second is a positive result that supplies the mechanism. Non-covalent planarizing interactions yield highly ordered and thermotropic liquid crystalline conjugated polymers uses weak intramolecular S···O and S···H–C contacts to keep the TPT-TT backbone flat at room temperature. These contacts release at high temperature, which keeps the polymer processable. On cooling, the polymer passes through a nematic phase (218–107 °C) and then a smectic phase (107–48 °C) before it crystallizes. Annealing through these stages tripled doped conductivity to 55 S/cm and gave 100% transistor yield. The authors also report a known weakness: small errors in the thermal schedule trap disorder. We take three design lessons from this paper. The locks should be local. They should be reversible. Order should be built in stages, with coarse alignment before fine alignment. We also take the warning that a schedule fixed in advance is fragile, and that warning motivates the feedback control. Neither paper is evaluated or extended here. The analogy only chooses the design, and every claim below is tested on image benchmarks.

Core Claim

The identity–editability trade-off in consistent generation is mostly a scheduling problem, not a strength problem. Suppose cross-image coupling is limited to corresponding subject regions and its strength follows cross-frame subject agreement measured during denoising. Then one method can reach identity consistency at least as good as batch-wide attention sharing, with less attribute leakage and more pose diversity. The same mechanism extends to arbitrarily long stories that arrive one image at a time, because each new image couples only to a small, fixed bank of already-generated anchor images.

Research Program

The mechanism. We generate $N$ images of one character with $N$ scene prompts, in parallel on a pretrained SDXL backbone. We use SDXL because it matches the comparisons in OneActor and StyleAligned. At denoising step $t$, each track produces a predicted clean image $\hat{x}_0^{(i)}$. The cross-attention map of the subject word gives a soft mask $m^{(i)}$ over the subject. We embed the masked subject crops with DINOv2 and define the order parameter $S_t$ as the mean pairwise cosine similarity of these embeddings across tracks. The coupling strength is then

$$\lambda_t = \sigma\big(\kappa\,(S_t - s^\ast)\big)\cdot g(t),$$

Here $\sigma$ is the logistic function, $s^\ast$ is the agreement threshold at which locking begins, $\kappa$ controls how sharp the transition is, and $g(t)$ is a coarse envelope that is zero at the very highest noise levels. The coupling itself adds to each track's self-attention a set of keys and values taken from the other tracks. These extra keys and values come only from locations inside the other tracks' subject masks, and they are weighted by $\lambda_t$. Because $S_t$ is recomputed at every step, the lock releases whenever the frames disagree, which is the reversible part of the design.

Staged ordering. Following the nematic-then-smectic sequence, $\lambda_t$ is applied first in the low-resolution attention layers of the network, which carry silhouette, palette and proportions. It moves to the high-resolution layers, which carry facial detail and texture, only once $S_t$ passes a second threshold. The ablations test whether this staging matters or whether one global schedule is enough.

Streaming. For long stories we keep an anchor bank of $K$ images, with $K$ between 2 and 4. The range comes from the observation in Group Diffusion Transformers are Unsupervised Multitask Learners that joint attention works for groups of 2 to 4 images and degrades at 8. Each new prompt is generated by coupling to the anchors only, with the anchor features cached at matched noise levels. The cost per image is therefore constant, and the method needs no access to future prompts.

Analysis. The central question is whether the ideal lock onset differs between prompts. We will measure it per prompt set by sweeping fixed onsets and recording which onset maximizes a combined identity and text-alignment score. If the spread is narrow, the feedback controller has no job to do, and the whole idea reduces to a tuned fixed window.

Baselines

The naive reference point is plain SDXL with a shared subject description and different seeds, with no coupling. It sets the floor on identity consistency and the ceiling on prompt freedom. Every method must beat its identity score while keeping text alignment within a small margin of it.

The strongest training-free competitor is BIPE from Boost the Identity-Preserving Embedding for Consistent Visual Generation. BIPE rescales the singular values of the shared prompt's text embeddings to amplify the common identity signal, and it concatenates text keys across frames. The paper reports that BIPE ranked first among training-free methods for identity consistency on ConsiStory+, had the highest text alignment, and stayed stable past 50 images. The summary gives no numeric CLIP-I or CLIP-T values, so there is no reported result found on Lacuna for BIPE's ConsiStory+ scores. We will rerun BIPE with its released settings on the same prompts, seeds and backbone. BIPE also needs all prompts in advance, which is exactly the streaming weakness our anchor bank targets. For streaming comparisons we give BIPE the full story up front, which is the most favourable setting for BIPE.

StyleAligned, from Style Aligned Image Generation via Shared Attention, shares keys and values from one reference track and normalizes queries and keys to limit copying of content. It reported a 67.1% human preference over StyleDrop and 61.3% over DreamBooth-LoRA. It is the closest attention-based relative of our method, and it lets us test fixed full-image sharing against masked, feedback-controlled sharing.

OneActor, from OneActor: Consistent Subject Generation via Cluster-Conditioned Guidance, tunes a small projector in 3–6 minutes. It is reported to sit on the Pareto front of image similarity (CLIP-I) against text alignment (CLIP-T), but no numeric values are given on Lacuna. It is the strongest method that uses tuning. We also include The Chosen One: Consistent Characters in Text-to-Image Diffusion Models, which clusters generated images and then fine-tunes on the most cohesive cluster, because it is the documented case of backgrounds getting baked into the weights. Both run at their published tuning budgets. We compare against them on full trade-off curves, not single points: OneActor's curve comes from its semantic-interpolation knob, and ours comes from sweeping $s^\ast$.

Beating these baselines means that our curve of masked-DINOv2 identity against CLIP-T lies strictly above BIPE's and StyleAligned's on DiverStory, the harder natural-language benchmark. It also means matching them on ConsiStory+ while showing lower attribute leakage and higher pose diversity at equal identity scores.

The ablations isolate the key ingredient, which is feedback. The main ablation replaces $\lambda_t$ with the best fixed onset-and-offset window, chosen by grid search on a held-out prompt split. The grid search gets the same number of evaluations we spend tuning $s^\ast$ and $\kappa$. Further ablations remove the subject mask (full-image sharing), remove the staging (all layers at once), and remove reversibility (once $\lambda_t$ rises it cannot fall).

Why not just…?

Why not just tune a fixed timestep window for attention sharing? This is the real threat, and the proposal is built so that it can lose to this alternative. If the per-prompt analysis shows that the ideal lock onset is nearly constant, a well-tuned fixed window will match the feedback controller and we will report that result. We expect a spread for two reasons. BIPE itself degrades when moving from rigid templates to the varied phrasing in DiverStory. And the scheduling fragility documented for TPT-TT is the physical version of this failure. The matched-budget grid-search ablation settles the question directly.

Why not just use BIPE, which is already training-free and scales past 50 images? BIPE works on text embeddings and text keys. It cannot see whether the images themselves agree, and it needs the whole story in advance. Our streaming experiment adds prompts one at a time. That setting is closed to BIPE by construction, so there we compare against BIPE given full foresight, and the question becomes whether a 2–4 image anchor bank loses anything. The two methods are also compatible. We will run BIPE embeddings plus our locks, and if the combination is best, our method is a complement to BIPE rather than a replacement.

Why not just fine-tune (LoRA or OneActor) and turn guidance down when editing? Tuning methods fix identity in weights that apply everywhere. The file records their failure modes. OneActor struggles to drop an umbrella or headgear in later scenes, and The Chosen One absorbs backgrounds into its LoRA. Lowering guidance weakens identity and accessories together, because the tuned weights contain both. Our accessory-removal test, described below, measures whether masked, reversible locks can keep the face while dropping the object. If OneActor's trade-off curve already dominates ours on that test, then the lesson from the polymers adds nothing and we will say so.

Why bring polymers in at all? They are not needed to evaluate the method, only to choose it. They supplied three specific design choices that the current consistency literature does not combine: local locks, reversible locks, and ordering in stages. They also supplied one specific risk, the quench trap. Each of these ideas maps to an ablation above, so the analogy either earns its place in the results or is shown to be decorative.

Proposed Experiments

The main benchmarks are ConsiStory+ (template prompts) and DiverStory (200 natural-language prompt sets), both from the BIPE paper, together with OneActor's category-diverse subject sets covering people, animals and objects. Identity is measured by DINOv2 similarity restricted to the subject mask, so that shared backgrounds do not inflate the score, with CLIP-I also reported. Text fidelity is CLIP-T. "Processability" is measured as pose and layout diversity, using variance of body keypoints and of subject bounding boxes across a story at a fixed identity score.

We add a leakage suite built from the failure cases in the file. A character is introduced with an accessory, such as an umbrella or a hat, or with a distinctive background, and later prompts ask for its removal. We score removal success with a vision-language judge and spot-check it by hand.

For scale we generate stories of 4, 8, 16 and 50 images in streaming mode and plot identity against position in the story. The plot shows whether anchor-bank coupling avoids the degradation that GDT reports at group size 8.

As a stress test we generate keyframes from MovieBench shot descriptions using its character banks, scored with its identity-precision metric (PrecisionID). The paper reports 41.3% PrecisionID for the best evaluated video models (MovieBench: A Hierarchical Movie Level Dataset for Long Video Generation). That figure comes from full video generation, so our keyframe results will be reported beside it for context, not as a like-for-like win.

The decisive checks come in order.

The first is the spread of ideal fixed onsets across prompts. If it is narrow, the core claim is dead before anything else runs.

The second is the matched-budget comparison of feedback control against the tuned fixed window on DiverStory. Our method must win on the identity-versus-text curve for the main claim to hold.

The third is the leakage suite against OneActor and The Chosen One. It tests whether masked, reversible coupling actually releases attributes that tuned weights keep.

The fourth is streaming against BIPE given full foresight. It tests whether a constant-size anchor bank can match a method that sees the whole story.