SerialGen: Personalized Image Generation by First Standardization Then Personalization

Cong Xie, Han Zou, Ruiqi Yu, Yan Zhang, Zhenpeng ZhanCVPR 2025 · DOI

Abstract

Personalized image generation aims to synthesize specific characters in diverse, user-defined scenarios. However, existing methods often suffer from a "tug-of-war" between text controllability and appearance consistency. We introduce SerialGen, a novel two-stage framework that standardizes reference images before personalization. By first transforming a character into a neutral pose and background, the model learns to extract robust identity features—including clothing and hairstyle—while maintaining the flexibility to follow complex text prompts. Experimental results show that SerialGen outperforms state-of-the-art methods in both identity preservation and prompt adherence.

Introduction: The Personalization Trade-off

The goal of personalized AI image generation is to take a reference image of a character and place them in new settings (e.g., "this character walking a dog in a park"). This task is notoriously difficult due to a fundamental trade-off. If a model is overly sensitive to the reference image, it often suffers from replication issues, where it copies the original pose or background, effectively ignoring the text prompt. Conversely, if the model prioritizes the text prompt, it often loses appearance consistency, altering the character's clothing, hair color, or body shape to fit the new context.

Most current approaches attempt to solve this by training on pairs of images derived from the same source through simple augmentations like cropping or flipping. This "easy" task allows models to "cheat" by copying pixels rather than understanding the character's identity. Furthermore, reference images often contain "Non-Appearance Elements" (NAE)—such as awkward poses or busy backgrounds—that distract the feature extractor from the character's actual appearance.

Methodology: The SerialGen Framework

SerialGen addresses these challenges by breaking the generation process into two distinct, sequential stages: Standardization and Personalization.

Stage I: Standardization

The objective of the first stage is to convert any arbitrary reference image into a "standardized" format: a neutral T-pose against a plain white background with a fixed facial scale. This process removes distracting NAEs and provides a clean template for feature extraction. To achieve high-quality standardization, the authors introduced two specialized modules:

  • Foreground-Background Distinction Module (FBDM): This module explicitly integrates a background mask, allowing the model to better distinguish the character from their original environment.
  • Reference Pose Injection Module (RPIM): By injecting pose information extracted via DWPose, the model can accurately locate and transform body parts into the target neutral pose.
Reference anime character image

Stage II: Personalization

Once a standardized reference is obtained, it is used to train the final generation model. Because the standardized image (neutral pose, white background) looks fundamentally different from the target image (dynamic pose, complex background), the model is "forced" to learn the character's intrinsic appearance—such as specific clothing patterns, hair textures, and facial features—rather than simply copying the input. This stage utilizes a frozen Stable Diffusion XL (SDXL) backbone with an architecture inspired by IP-Adapter.

Data and Evaluation

To train a robust standardization model, the researchers created a massive synthetic dataset. They rendered 2,924 3D character models in 3,817 unique poses, generating over 10.5 million synthetic image pairs. For the personalization stage, they processed approximately 300,000 real-world character images through their Stage I model to create a large-scale dataset of (standardized reference, original target) pairs.

The model was evaluated on a curated test set of 40 characters (split evenly between anime and real-life styles) paired with 20 diverse text prompts generated by ChatGPT-4 to ensure a wide range of testing scenarios.

Key Results and Discussion

SerialGen demonstrates significant improvements over existing state-of-the-art models like IP-Adapter, FastComposer, and StoryMaker across both quantitative and qualitative benchmarks.

  • Text Alignment (CLIP-T): SerialGen achieved a score of 21.76, surpassing IP-Adapter (20.22) and StoryMaker (20.45). This indicates that the model is better at following complex instructions without being "locked" into the reference image's original state.
  • Image Similarity (CLIP-I): The model achieved a score of 85.49, the highest among all compared methods, proving its superior ability to maintain appearance consistency.

Qualitatively, SerialGen excels at preserving "whole-body appearance." While other models might change a character's outfit color or hair length when asked to perform a specific action, SerialGen maintains the exact features from the reference.

Generated result: character walking dogs

Conclusion and Future Directions

SerialGen's "standardize-then-generate" workflow provides a powerful solution to the long-standing conflict between controllability and consistency in image personalization. By treating clothing and hairstyle as integral parts of a character's identity and removing distracting environmental elements, it achieves a new level of fidelity. Future research may explore extending this framework to video generation, where maintaining character consistency across frames remains a significant challenge.

Summary updated

BibTeX
@inproceedings{xie2025serialgen,
  author = {Cong Xie and Han Zou and Ruiqi Yu and Yan Zhang and Zhenpeng Zhan},
  title = {{SerialGen: Personalized Image Generation by First Standardization Then Personalization}},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year = {2025},
  doi = {10.1109/cvpr52734.2025.00271},
}
Figure