ConceptWeaver: Weaving Disentangled Concepts with Flow

Jintao Chen, Aiming Hao, Xiaoqing Chen, Chengyu Bai, Chubin Chen, Yanxun Li, Jiahong Wu, Xiangxiang Chu, Shanghang ZhangarXiv 2026 · arXiv

TL;DR

ConceptWeaver introduces a novel differential probing technique that uncovers a universal, three-stage generative process (Blueprint, Instantiation, and Refinement) inside flow-based diffusion models. By exploiting the fact that visual concepts naturally disentangle during a specific mid-generation window, the framework achieves state-of-the-art, one-shot concept personalization and composition without architectural lock-in or text-control degradation.

What Problem Does This Paper Address?

Personalizing text-to-image and text-to-video models—tuning them to generate a specific object, style, or motion based on just a single reference image—is notoriously difficult. Existing methods often suffer from the "copy-paste" phenomenon, where the model rigidly copies the background or pose of the reference image instead of cleanly extracting the core concept.

Furthermore, current state-of-the-art personalization techniques are often constrained by architecture (e.g., relying on specific text-modulation pathways found only in certain models) and completely ignore the temporal dimension of generation. They apply guidance uniformly across the entire denoising process, failing to consider when specific features actually form.

Methods

ConceptWeaver tackles these challenges by first analyzing how concepts emerge over time using a new technique called Differential Probing. By measuring the difference in the model's velocity field when a specific text token is present versus when it is masked, the authors map out the exact lifecycle of individual concepts.

This probing revealed that flow-based generation consistently unfolds in three distinct stages:

  1. Blueprint Stage (Early timesteps): The model outlines low-frequency, coarse structural layouts and motions.
  2. Instantiation Stage (Mid timesteps): Content concepts (like identities or specific objects) peak sharply in intensity and naturally disentangle from the rest of the scene.
  3. Refinement Stage (Late timesteps): The model becomes insensitive to high-level concepts and focuses on fine-grained textures and global coherence.
Fig. 2: Probing Concept Formation Dynamics. While standard prompt guidance (a) provides a dense, entangled signal, our differential probing technique (b) isolates the dynamic influence of individual concepts. This analysis uncovers a consistent three- stage framework (c): an early Blueprint Stage dominated by structural concepts (e.g.,“riding a bike”), a pivotal mid-stage Instantiation Stage where content concepts (e.g., “panda”) are decoupled and peak in intensity, and a final Refinement Stage where signals fade.

Using this insight, the authors built ConceptWeaver, which operates in two phases:

  • Stage-Aware Optimization: Instead of training uniformly, a lightweight Semantic Offset Module learns concept-specific semantic offsets ($\Delta e_i$) within the cross-attention layers. Training is concentrated strictly during the concept's peak-influence stage (e.g., the Instantiation stage for objects, or the Blueprint stage for motion).
  • ConceptWeaver Guidance (CWG): During inference, these learned offsets are strategically injected only during their corresponding generative stages rather than throughout the entire process.
Fig. 3: Flowchart of proposed ConceptWeaver. Our framework consists of two main phases: training and inference. (a) During training, a lightweight Semantic Offset Module learns to represent a visual concept by adding a learnable offset to the key/value pairs of its corresponding concept token. This is optimized with a stage-aware loss. (b) During inference, our ConceptWeaver Guidance (CWG) applies these offsets in a stage-aware manner: structural offsets are injected during the Blueprint Stage, and content offsets during the Instantiation Stage, enabling precise compositional control.

Datasets

The authors evaluated their approach using several datasets:

  • Single-Concept Personalization: 20 images extracted from Dreambench++ and DreamBooth, supplemented by 10 motion datasets collected from the internet.
  • Multi-Concept Composition: 20 images sourced from the internet containing distinct subjects and attributes.
  • Processing: All self-collected data was automatically captioned using a Large Multimodal Model (LMM), and an LLM was used to generate diverse evaluation prompts.

Key Insights and Ideas

  • Generation is Not Monolithic: Flow-based models do not generate all parts of a prompt simultaneously. The discovery of the three-stage framework proves that structure, content, and texture have strict, independent temporal windows.
  • The Instantiation Window is Optimal for Control: Because content concepts naturally disentangle and peak in intensity during the middle of the generation process, this stage provides a highly effective, principled window for injecting personalized attributes without causing structural artifacts.
  • Architecture-Agnostic Personalization: By shifting the locus of control to cross-attention key/value pairs and aligning it with temporal dynamics, ConceptWeaver avoids the need for specialized text-modulation paths, making it universally applicable to various flow models (e.g., Flux, Wan2.2, Stable Diffusion 3).

Key Results

Evaluated on the base architecture of Wan2.2-TI2V-5B, ConceptWeaver achieved state-of-the-art performance in single-concept generation, outperforming prominent baselines like DreamBooth and MS-Diffusion:

  • Textual Alignment ($S^T_{\text{CLIP}}$): Achieved a top score of 35.22.
  • Concept Alignment: Led across the board with a CLIP image similarity ($S^I_{\text{CLIP}}$) of 76.13, a DINO score of 69.57, and an MLLM GPT-4o evaluation score of 3.8/4.0.

Qualitatively, the method excels at multi-concept composition (e.g., putting a specific style of sunglasses onto a specific dog) while successfully avoiding pose overfitting and concept bleeding. It also enables unique applications like Motion Concept Transfer—extracting the motion from a reference video (during the Blueprint stage) and applying it to an entirely new subject.

Limitations and Future Work

  • Base Model Dependency: The performance of ConceptWeaver scales directly with the foundational capabilities of the base generative model.
  • Semantic Confusion: When prompts contain highly similar concepts (e.g., trying to generate a "dog" and a "doll" together), the model can occasionally suffer from semantic infiltration, blending the two concepts undesirably.
  • Complex Physical Actions: The framework struggles if the target task exceeds the base model's physical reasoning capabilities, such as depicting a rabbit realistically executing human-like bicycle-riding motions.
  • Future Direction: The authors note that while they currently categorize concepts (structural vs. content) manually, a promising future step is automating the discovery of these temporal profiles.

Summary updated

Abstract

Pre-trained flow-based models excel at synthesizing complex scenes yet lack a direct mechanism for disentangling and customizing their underlying concepts from one-shot real-world sources. To demystify this process, we first introduce a novel differential probing technique to isolate and analyze the influence of individual concept tokens on the velocity field over time. This investigation yields a critical insight: the generative process is not monolithic but unfolds in three distinct stages. An initial \textbf{Blueprint Stage} establishes low-frequency structure, followed by a pivotal \textbf{Instantiation Stage} where content concepts emerge with peak intensity and become naturally disentangled, creating an optimal window for manipulation. A final concept-insensitive refinement stage then synthesizes fine-grained details. Guided by this discovery, we propose \textbf{ConceptWeaver}, a framework for one-shot concept disentanglement. ConceptWeaver learns concept-specific semantic offsets from a single reference image using a stage-aware optimization strategy that aligns with the three-stage framework. These learned offsets are then deployed during inference via our novel ConceptWeaver Guidance (CWG) mechanism, which strategically injects them at the appropriate generative stage. Extensive experiments validate that ConceptWeaver enables high-fidelity, compositional synthesis and editing, demonstrating that understanding and leveraging the intrinsic, staged nature of flow models is key to unlocking precise, multi-granularity content manipulation.
BibTeX
@misc{chen2026conceptweaver,
  author = {Jintao Chen and Aiming Hao and Xiaoqing Chen and Chengyu Bai and Chubin Chen and Yanxun Li and Jiahong Wu and Xiangxiang Chu and Shanghang Zhang},
  title = {{ConceptWeaver: Weaving Disentangled Concepts with Flow}},
  year = {2026},
  eprint = {2603.28493},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2603.28493},
}
Figure