PoseMaster: A Unified 3D Native Framework for Stylized Pose Generation

Hongyu Yan, Kunming Luo, Weiyu Li, Kaiyi Zhang, Yixun Liang, Jingwei Huang, Chunchao Guo, Ping Tan

TL;DR

PoseMaster is a unified, 3D-native generative framework that achieves precise 3D character pose stylization directly from a single reference image and a target 3D skeleton. By replacing traditional multi-stage 2D-to-3D cascade pipelines with a single-stage flow-based Diffusion Transformer, it eliminates error propagation and depth ambiguity. Crucially, because the generated 3D meshes strictly align with the input 3D skeleton, PoseMaster enables the immediate creation of fully rigged, animatable assets.

What Problem Does This Paper Address?

Generating stylized 3D characters that match a specific target pose while preserving the visual identity of a reference image is a major bottleneck in gaming, film, and VR.

Existing state-of-the-art methods typically rely on a decoupled cascade pipeline:

  1. Use a 2D controllable model (like ControlNet) guided by a 2D skeleton image to manipulate the character's pose in 2D space.
  2. Use a separate 3D reconstruction model to "lift" that generated 2D image into a 3D asset.

This approach suffers from two fundamental flaws. First, error propagation is inevitable—any artifacts, warping, or self-occlusion mistakes made by the 2D model are permanently baked into and amplified by the 3D reconstruction. Second, 2D skeleton projections introduce geometric ambiguity, as flattening a pose into 2D strips away critical depth and topological information.

PoseMaster bypasses these limitations by introducing a 3D-native framework that handles pose control and 3D generation simultaneously.

Figure 1. Given a single image and arbitrary poses represented by a 3D skeleton, PoseMaster can generate a high-quality 3D asset that maintains the identity of the image while adhering to the pose defined by the skeleton, enabling rich and precise 3D pose stylization.

Methods

PoseMaster unifies pose stylization and 3D generation into an end-to-end architecture consisting of a 3D Variational Autoencoder (VAE) and a flow-based Diffusion Transformer (DiT), built upon the Hunyuan3D 2.1 backbone.

Figure 2. Overall pipeline of our PoseMaster. Given a single image and a 3D skeleton, our PoseMaster integrates them into a unified 3D native generation framework to achieve precise pose stylization under 3D generation.

The core technical innovations include:

  • 3D Skeleton Control Mechanism: Instead of 2D maps, the model uses explicit 3D spatial coordinates.
  • Skeleton Densification Strategy: Standard 3D skeletons only define sparse joints, which lacks topological connectivity. PoseMaster uses a distance-weighted interpolation scheme to transform bones into a dense point cloud. It then embeds explicit 3D bone directional vectors into these points to give the network structural awareness.
  • Point Transformer Skeleton Encoder: This module processes the densified coordinates and directional features into a high-dimensional skeleton condition token ($c_p$).
  • Multi-Condition DiT: The model injects both the image condition ($c_i$, extracted via DINOv2) and the skeleton condition ($c_p$) at the token level into the DiT block, using Classifier-Free Guidance (CFG) to balance identity preservation and pose adherence.
Figure 4. The skeleton representation of sparse joints and ours. We propose a distance-weighted interpolation scheme to densify the skeletal graph. Furthermore, explicit bone directional vectors are embedded into all interpolated points belonging to the same skeletal segment.

Datasets

Training a 3D-native model requires massive amounts of paired data, which is notoriously scarce. To solve this, the authors built a scalable Data Engine to curate a large-scale dataset of "Image-Skeleton-Mesh" triplets:

  • Animatable 3D Assets: Sourced from ReadyPlayerMe, VRoid, and Playbox. Characters were animated using motion sequences to create Action Pairs—sampling an image from one frame and pairing it with the mesh and skeleton from a completely different frame to force pose decoupling.
  • Static 3D Meshes: Sourced from Objaverse, Objaverse-XL, and HumanRig. For unrigged models, an automatic rigging model was used to extract skeletons. Multi-view renderings were generated to create View Pairs, forcing the network to infer occluded regions.

In total, the data engine produced data for over 500K unique humanoid objects, scaling into millions of total training pairs.

Overview of the dataset construction pipeline.

Key Insights and Ideas

  • The 3D Skeleton as a Geometric Anchor: Directly feeding a 3D skeleton into a 3D native generator acts as an explicit structural scaffold. This resolves the monocular depth ambiguity that plagues standard image-to-3D models, preventing geometric collapse in complex or self-occluded poses.
  • Bypassing the 2D Intermediate Step: By eliminating the intermediate 2D image-editing step, the model avoids the structural distortions common to 2D models, resulting in significantly cleaner mesh topologies.
  • Downstream Rigging for Free: Because the generated 3D mesh is synthesized to strictly conform to the input 3D skeleton's spatial layout, the asset is inherently pre-aligned. When paired with standard automated skinning models, it can be animated immediately, completely skipping manual skeleton retargeting.

Key Results

The authors evaluated PoseMaster on two primary tasks: pose canonicalization (turning an arbitrary pose into a standard T/A-pose) and arbitrary-pose stylization.

Quantitative Performance

Evaluated using Mean Angular Error (MAE) and Cosine Similarity (Sim) of surface normals to circumvent spatial misalignment penalties:

  • Pose Canonicalization (VRoid dataset): PoseMaster achieved an MAE of 4.59 and Sim of 0.938, outperforming task-specific pipelines like StdGen (MAE 4.97) and native generators like Hunyuan3D 2.1 (MAE 5.89).
  • Arbitrary-Pose Stylization: PoseMaster achieved an MAE of 5.28, significantly lower than Hunyuan3D 2.1 (6.75) and Trellis (7.20), even when those baselines were given the ground-truth target image as input.

Efficiency

Because it is a single-stage framework, PoseMaster cuts down intermediate computational bottlenecks, achieving an inference time of ~23.48 seconds on a single NVIDIA H20 GPU, compared to StdGen's ~61.54 seconds.

Figure 5. The qualitative comparison for pose canonicalization on VRoid [7] dataset. Our method achieves higher geometry quality and accuracy compared with baselines.

Limitations and Future Work

While PoseMaster demonstrates strong overall pose controllability, the authors note a few limitations:

  • Fine-Grained Details: The model still struggles to synthesize highly intricate geometric details, such as complex hairstyles, flowing skirts, or precise finger/hand gestures.
  • Resolution Limits: The current single-stage generation paradigm operates at a spatial resolution of 512. As a result, its high-frequency geometric fidelity can lag behind multi-stage refinement methods.

The authors suggest that integrating a high-resolution geometric refinement module in future work will be key to upscaling the coarse meshes and achieving photorealistic details.

Summary updated

Abstract

Pose stylization, which aims to synthesize stylized content aligning with target poses, serves as a fundamental task across 2D, 3D, and video domains. In the 3D realm, prevailing approaches typically rely on a cascade pipeline: first manipulating the image pose via 2D foundation models and subsequently lifting it into 3D representations. However, this paradigm limits the precision and diversity of the 3d pose stylization. To this end, we propose a novel paradigm for 3D pose stylization that unifies pose stylization and 3D generation within a cohesive framework. This integration minimizes the risk of cumulative errors and enhances the model's efficiency and effectiveness. In addition, diverging from previous works that typically utilize 2D skeleton images as guidance, we directly utilize the 3D skeleton because it can provide a more accurate representation of 3D spatial and topological relationships, which significantly enhances the model's capacity to achieve richer and more precise pose stylization. Moreover, we develop a scalable data engine to construct a large-scale dataset of ''Image-Skeleton-Mesh'' triplets, enabling the model to jointly learn identity preservation and geometric alignment. Extensive experiments demonstrate that PoseMaster significantly outperforms state-of-the-art methods in both qualitative and quantitative metrics. Owing to the strict spatial alignment between the generated 3D meshes and the conditioning skeletons, PoseMaster enables the direct creation of animatable assets when coupled with automated skinning models, highlighting its compelling potential for automated character rigging.
BibTeX
@inproceedings{yan2026posemaster,
  author = {Hongyu Yan and Kunming Luo and Weiyu Li and Kaiyi Zhang and Yixun Liang and Jingwei Huang and Chunchao Guo and Ping Tan},
  title = {{PoseMaster: A Unified 3D Native Framework for Stylized Pose Generation}},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year = {2026},
  url = {https://openreview.net/forum?id=bFOa064XDZ},
}
Figure