Abstract
Recent advances in text-to-image (T2I) diffusion models have enabled personalized image generation through techniques like DreamBooth. However, generating multiple customized concepts in a single image remains a significant challenge due to "feature confusion" and the computational overhead of joint training. We present MultiBooth, a two-phase framework that decouples single-concept learning from multi-concept integration. By utilizing a Regional Customization Module (RCM) and Adaptive Concept Normalization (ACN), MultiBooth achieves high-fidelity multi-concept customization in a plug-and-play manner, significantly reducing storage requirements and inference time while preventing feature bleeding.
Introduction
Text-to-image customization allows users to teach models specific, personal concepts using a few reference images. While effective for single subjects, Multi-Concept Customization (MCC) often fails when multiple subjects are combined. Existing joint-training methods frequently suffer from feature bleeding—where attributes of one subject (e.g., a teapot's texture) are incorrectly applied to another (e.g., a jar). Furthermore, these methods lack scalability, requiring a new model for every unique combination of subjects. MultiBooth addresses these issues by treating concepts as independent modules that can be integrated spatially at inference time.
Methodology
MultiBooth operates in two distinct phases to ensure clean concept separation and efficient integration.
Phase 1: Single-Concept Learning
Concepts are learned individually to avoid interference. A lightweight multi-modal encoder (QFormer) extracts clean features, filtering out irrelevant background noise. To address the "embedding domain gap"—where custom embeddings often have significantly higher L2 norms than standard CLIP embeddings—the authors introduce Adaptive Concept Normalization (ACN). ACN scales custom embeddings to match standard token magnitudes, ensuring balanced prompt influence. Finally, concepts are stored efficiently using Low-Rank Adaptation (LoRA).
Phase 2: Multi-Concept Integration
To combine concepts without retraining, MultiBooth introduces the Regional Customization Module (RCM). During inference, users provide bounding boxes for concept placement. The RCM modifies the U-Net’s cross-attention layers, ensuring that specific text embeddings and LoRA weights are applied only within their designated regions. Overlapping regions are managed via a weighted average strategy to facilitate natural interactions.
Key Results and Discussion
MultiBooth demonstrates superior performance across several benchmarks, including the DreamBooth and CustomConcept101 datasets.
- Performance Metrics: MultiBooth achieved state-of-the-art scores in CLIP-I (0.714) and CLIP-T (0.838), outperforming baselines like Textual Inversion and Custom Diffusion.
- Efficiency: Each concept module requires less than 7MB of storage. Training takes approximately 6 minutes per concept, and inference scales efficiently (e.g., ~10.5 seconds for 4 concepts).
- Qualitative Fidelity: The model successfully maintains the identity of multiple subjects even in complex scenes, avoiding the feature bleeding common in joint-training approaches.
Conclusion
MultiBooth provides a scalable and high-fidelity solution for multi-concept text-to-image generation. By decoupling the learning process and employing regional attention, it eliminates feature bleeding and the need for expensive joint retraining. Future work may explore automated layout generation to further simplify the user experience for complex scene composition.
Summary updated