Text-to-image diffusion models generate a picture by gradually denoising random noise under a text prompt. Ask one of them for "a wizard reading" and then "a wizard casting a spell" and you get two different wizards. Current fixes for this identity drift sit at two poles. Some bind identity rigidly: they swap in a fixed identity token, share every attention key and value with a reference image, or rescale the embedding matrix of the whole prompt sequence at once. These keep the face but fight the prompt when a scene legitimately changes the character. Others bind loosely and drift. Organic-semiconductor chemists faced the same trade-off when designing conjugated polymer backbones, which must stay flat to conduct charge. Fusing neighbouring rings with covalent bonds makes the polymer flat but insoluble and impossible to process. Their answer was weak intramolecular contacts, such as sulfur–oxygen attractions, that hold rings coplanar at operating temperature, let go when heated, and re-form on cooling (Non-covalent planarizing interactions yield highly ordered and thermotropic liquid crystalline conjugated polymers).
We propose to import the functional form of that solution, not just the metaphor. Each new image in a stream is attracted to a running identity memory through a bounded, well-shaped potential. The pull is strongest for small deviations and fades to zero for large ones, so a feature the prompt genuinely moves away (a new outfit, a helmet) breaks its lock instead of being dragged back. The lock strength also rises as the noise level falls. Because the memory is updated one image at a time, the method runs in a streaming setting, which the strongest training-free method cannot do.
Core Claim
We claim that a training-free, streaming identity lock with a saturating potential, gated by noise level, will reach a better trade-off between identity consistency and prompt adherence than both rigid locks and quadratic soft locks. The advantage should be largest on prompts that explicitly change some attribute of the character.
Concretely, at denoising step $t$ the sampler adds the gradient of
$$E_t(x)=\lambda(\sigma_t)\sum_k\Big[1-\exp\!\Big(-\tfrac{\|f_k(x)-m_k\|^2}{2s^2}\Big)\Big]$$
where:
- $x$ is the current latent and $\sigma_t$ is the noise level.
- $f_k(x)$ are identity features at site $k$. These are the subject token's identity component in the text embedding and the subject-region self-attention values at selected layers.
- $m_k$ is the stored memory for site $k$.
- $s$ is the lock length.
- $\lambda(\sigma)$ is a gate that is near zero at high noise, where layout is decided, and large at low noise, where features are resolved.
The gradient of each term peaks at distance $s$ and vanishes beyond it. That is the "breakable" property. A quadratic penalty, $\|f_k-m_k\|^2$, pulls hardest exactly when the prompt has moved a feature farthest.
After each image, the memory is updated only at sites that stayed locked ($m_k\leftarrow(1-\eta)m_k+\eta f_k$ when $d_k<s$). This mirrors the way the liquid-crystal polymer re-orders progressively as it cools, without processing the whole sample at once.
Where the chemistry constrains the design
The materials literature supplies three testable design priors.
First, locks dissociate thermally and re-form on cooling (TPT-TT liquid crystal paper). This motivates the noise gate $\lambda(\sigma)$ and predicts that ungated locks will distort composition.
Second, aldol-condensation polymers lose planarity and mobility when the regular, periodic locking motif is broken up in random copolymers (The Influence of Interlocking Effects in Conjugated Polymers Synthesized by Aldol Polycondensation). This motivates locking every frame to one shared memory, rather than chaining each frame to its predecessor, where errors compound irregularly. We test both.
Third, a one-ring change flips TPT-T from edge-on crystalline packing to helical packing (Non-Covalent Interactions and Helical Packing in Thiophene-Phenylene Copolymers). This warns that lock placement matters a great deal, and it motivates an explicit sweep over which layers and sites carry locks.
We do not run chemistry experiments. The chemistry is the design lens, and its predictions are what the ablations falsify. Near-miss papers that apply diffusion models to conformational ensembles (Unlocking hidden biomolecular conformational landscapes, Precise Generation of Conformational Ensembles for IDPs) go in the opposite direction, using generators for molecules. Non-Cross Diffusion targets general text–image semantic fidelity, not persistent identity.
Research Program
We build the sampler on SDXL. SDXL is the backbone used by MagicNaming and Style Aligned, which keeps comparisons direct.
Identity features at text level. We take the subject token's identity component from the text embedding. We follow BIPE's observation that identity concentrates in the leading singular directions (Boost the Identity-Preserving Embedding). Unlike BIPE, which decomposes the joint matrix of all prompts, we compute these directions against the memory alone, so no future prompts are needed.
Identity features at image level. We use subject-region self-attention values, localized by cross-attention maps of the subject token.
First image. The first image of a stream is generated freely and initializes the memory.
Released sites as a signal. We log which sites release their lock on each prompt. This gives an interpretable record of what the model judged the prompt to change, and we check it against the attributes the prompt actually edits.
Multiple subjects. We give each subject its own memory and potential, and add a repulsive bounded term between memories. This targets the cross-subject bleeding reported by CoCoIns and the failure beyond four subjects reported by OneActor.
Baselines
Do nothing. Vanilla SDXL with independent noise per prompt and identical prompts. This is the floor for identity similarity and the ceiling for prompt alignment.
Strongest existing method: BIPE. BIPE combines adaptive singular-value rescaling with union keys. It reports the highest text alignment and first place for identity consistency among training-free methods on ConsiStory+. However, no reported absolute numbers for BIPE on ConsiStory+ or DiverStory were found on Lacuna, so we rerun it ourselves.
BIPE sees the whole prompt set in advance and our method does not, so we treat BIPE as an offline upper reference. Matching it in streaming mode, or beating it on attribute-change prompts, counts as success. Both methods use the same SDXL checkpoint, the same seeds, the same 50 denoising steps, and an equal hyperparameter search budget of 20 configurations, chosen on a held-out prompt split.
Strongest streaming-compatible rigid lock: shared attention to the first image. We implement this in the style of Style Aligned, with adaptive instance normalization on queries and keys. Its reported user-study wins (67.1% over StyleDrop, 61.3% over DreamBooth-LoRA) are for style, not identity.
Learned fixed-token lock: MagicNaming. MagicNaming reports ID Consistency 0.5624 and CLIP-TI 38.24 on SDXL. We run it on our prompts from the first-frame face. It uses training data we do not use, so beating it on prompt alignment at equal identity counts as a win. Matching it on identity counts as parity.
Key ablations. The quadratic-potential variant isolates the saturating well, which is the core ingredient. The ungated variant ($\lambda$ constant) isolates noise gating. The chain-memory variant isolates locking to a shared anchor.
Why not just…?
Why not just use a fixed identity token like MagicNaming or CharacterFactory? A fixed token is already streaming-compatible, so it is the most serious threat. The weakness is that a fixed token offers no per-feature release. Either the whole identity binds or none of it does. CharacterFactory needed a dedicated context-consistent loss over 1,000 prompts just to keep the token stable across actions. Our attribute-change suite tests this directly. If MagicNaming matches us on both identity and attribute adherence, the bounded lock adds nothing, and we will report that.
Why not just share attention with the first image? Full key and value sharing is a rigid, covalent-style bond. It transfers clothing and context along with the face. The file notes that reference attributes leak into the scene for attention-sharing methods. The attribute-change prompts measure exactly this leakage.
Why not just use a quadratic penalty or a robust loss with a timestep schedule? This is the honest core of the question. Our potential is a robust (Welsch-type) loss, and the claim stands or falls on whether saturation measurably helps. The quadratic ablation at a matched tuning budget settles it. If the two tie on the identity–alignment frontier, the chemistry-derived ingredient is not doing work.
Why not just fine-tune per character (DreamBooth or LoRA)? It cannot run in a stream. CharacterFactory reports 25 minutes per subject for DreamBooth, and OneActor still needs 3–6 minutes of tuning. We include The Chosen One-style LoRA only as an offline identity reference.
Proposed Experiments
Benchmarks. We use ConsiStory+ and DiverStory, the 200 syntactically varied prompt sets from BIPE, with prompts fed in order one at a time. We report identity similarity (DINOv2 and CLIP-I on subject crops), face similarity using the CelebA protocol of CoCoIns, and CLIP-T. CoCoIns reports Sim 0.256 against StoryDiffusion's 0.159 on complex scenes; we use these as reference points.
New attribute-change suite. We add 300 prompts, each explicitly editing one attribute (outfit, hair, accessory, age). These are scored by a vision–language judge for whether the edit appears, and by face similarity for whether identity holds.
Multi-subject. We test two to six characters, with cross-subject leakage measured as the face similarity of character A's crop to character B's memory.
Application-level check. We run the IDEA-Bench text-to-multi-image subset. There, FLUX-1 scores 22.48/100 and SD3 14.83, but our SDXL base is not comparable to those scores, so we compare only within SDXL.
Decisive check 1. On the attribute-change suite, the bounded lock should beat both the quadratic lock and first-image attention sharing on edit adherence, at equal face similarity.
Decisive check 2. Ungated locks should lower CLIP-T on pose-heavy prompts. This is the dissociation prediction.
Decisive check 3. Chain memory should drift more than anchor memory after 50 frames. This is the periodicity prediction.
If all three fail, the polymer analogy offers no usable design guidance and we will say so.
References
- Non-covalent planarizing interactions yield highly ordered and thermotropic liquid crystalline conjugated polymers
- The Influence of Interlocking Effects in Conjugated Polymers Synthesized by Aldol Polycondensation on Field-Effect Transistor Properties and Morphology
- Non-Covalent Interactions and Helical Packing in Thiophene-Phenylene Copolymers: Tuning Solid-State Ordering and Charge Transport for Organic Field-Effect Transistors
- Unlocking hidden biomolecular conformational landscapes in diffusion models at inference time
- Precise Generation of Conformational Ensembles for Intrinsically Disordered Proteins via Fine-tuned Diffusion Models
- Non-Cross Diffusion for Semantic Consistency
- MagicNaming: Consistent Identity Generation by Finding a "Name Space" in T2I Diffusion Models
- Style Aligned Image Generation via Shared Attention
- Boost the Identity-Preserving Embedding for Consistent Visual Generation
- CoCoIns: Consistent Subject Generation via Contrastive Instantiated Concepts
- OneActor: Consistent Subject Generation via Cluster-Conditioned Guidance
- CharacterFactory: Sampling Consistent Characters With GANs for Diffusion Models
- IDEA-Bench: How Far are Generative Models from Professional Designing?