IDOL: Instant Photorealistic 3D Human Creation from a Single Image

Yiyu Zhuang, Jiaxi Lv, Hao Wen, Qing Shuai, Ailing Zeng, Hao Zhu, Shifeng Chen, Yujiu Yang, Xun Cao, Wei Liu

Introduction

Creating a realistic 3D human from a single 2D image is a fundamental challenge in computer vision. The infinite variations in human pose, clothing, and body shape make this an ill-posed problem, further complicated by the scarcity of large-scale, diverse training data. Existing methods typically struggle to balance speed and quality: optimization-based approaches yield high detail but are slow, while feed-forward models are fast but often lack realism and fail to generalize to "in-the-wild" scenarios.

IDOL (Instant Photorealistic 3D Human Creation) addresses these limitations with a feed-forward transformer framework capable of generating high-fidelity, animatable 3D avatars in under one second.

Methodology

The IDOL architecture leverages three core innovations to achieve its performance:

  1. Sapiens Encoder: The model utilizes Sapiens, a high-resolution foundation model pretrained on 300 million human images. This specialized encoder allows for the extraction of fine-grained features from 1024x1024 input images, capturing nuances like fabric folds and facial details.
  2. Structured 3D Gaussian Representation: IDOL represents the human body using 3D Gaussian Splatting (3DGS). To optimize computation, these Gaussians are mapped into a structured 2D UV space defined by the SMPL-X body model. The network predicts attribute maps (offsets, color, opacity) within this 2D grid.
  3. UV-Alignment Transformer: This module bridges the gap between irregular image features and the regular UV grid, ensuring precise alignment between the 3D geometry and the input person's appearance.
Overview of the IDOL framework, including the pipeline, dataset, and generalizability results.

By tying the Gaussians to an underlying SMPL-X template, the generated avatars are inherently animatable via standard linear blend skinning (LBS) without additional post-processing.

The HuGe100K Dataset

A significant contribution of this work is the introduction of HuGe100K, a massive synthetic dataset designed to overcome data scarcity.

  • Scale: It features 100,000 diverse subjects and 2.4 million high-resolution multi-view images.
  • Pipeline: The dataset was created using GPT-4 for diverse prompting, Flux for base image generation, and a custom video model called MVChamp for consistent 360-degree multi-view synthesis. This scale allows IDOL to handle complex poses and diverse clothing styles (such as kimonos or loose coats) that previous models could not reconstruct accurately.

Results and Applications

IDOL demonstrates state-of-the-art performance in both speed and quality. It generates a full 3D avatar with 1K resolution textures in approximately 0.6 seconds on an NVIDIA A100 GPU. Quantitatively, it significantly outperforms existing baselines; for instance, on the 2K2K dataset, IDOL achieved a PSNR of 21.67 compared to the previous state-of-the-art GTA's 14.28.

Demonstration of IDOL's applications in character animation and texture editing.

Beyond reconstruction, the structured UV representation enables seamless downstream applications. Avatars can be instantly animated with new motion sequences or edited by modifying the underlying UV texture maps, such as adding custom graphics to clothing.

Limitations

While IDOL achieves high-fidelity results, it currently lacks a specialized sub-network for fine-grained facial expressions, treating the face as part of the general UV map. Additionally, the model performs best with full-body inputs; performance may degrade when reconstructing subjects from half-body images where lower-body cues must be inferred without sufficient visual evidence.

Summary updated

BibTeX
@inproceedings{zhuang2025idol,
  author = {Yiyu Zhuang and Jiaxi Lv and Hao Wen and Qing Shuai and Ailing Zeng and Hao Zhu and Shifeng Chen and Yujiu Yang and Xun Cao and Wei Liu},
  title = {{IDOL: Instant Photorealistic 3D Human Creation from a Single Image}},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year = {2025},
  doi = {10.1109/cvpr52734.2025.02450},
}
Figure