Abstract
Generating realistic 3D human avatars that interact naturally with indoor environments is a complex task requiring precise scene-aware interaction. This paper presents a novel framework for generating 3D humans using a pose-guided geometric alignment strategy. By integrating a pose classifier within a Conditional Variational Autoencoder (cVAE), the model adaptively applies geometric constraints to relevant body parts. This approach ensures physically plausible contact—such as thighs touching a seat or a back resting on a bed—while significantly increasing pose diversity and eliminating the need for costly post-processing optimization.
Introduction
The placement of 3D human meshes in digital environments requires more than simple spatial positioning; it demands realistic physical interaction. A common failure in existing generation methods is the lack of "scene-awareness," where avatars appear to float above surfaces or interpenetrate (clip) through furniture. Traditional solutions often rely on global contact rules or heavy post-hoc optimization, which can stifle the variety of generated poses and increase computational overhead. The core challenge lies in developing a model that understands how different activities, such as sitting or lying down, necessitate distinct physical contact points with the environment.
Methodology
The proposed framework is built upon a Conditional Variational Autoencoder (cVAE) designed to generate human meshes conditioned on scene geometry. The innovation centers on two primary components:
- Pose Classifier: A compact MLP-based network that categorizes generated poses into four classes: standing, sitting, lying, or ambiguous.
- Adaptive Geometric Alignment Loss ($L_{geo}$): This loss function is applied selectively based on the pose classification. It consists of a point-wise distance metric (Chamfer distance) to pull specific body parts toward surfaces and a surface normal alignment constraint. The latter ensures that the orientation of the body mesh matches the contact surface, preventing clipping and ensuring a "flush" fit.
By training the model to "snap" appropriate body segments to the environment, high-quality and physically plausible results are achieved directly during the inference stage.
Datasets
The research utilizes several key benchmarks:
- PROX and PROX-E: Standard datasets for 3D human-scene interaction.
- PROX-P (PROX-Pose): A new dataset introduced by the authors, featuring 12.5k manually annotated frames with pose labels and refined body segmentation (26 parts vs. the standard 8) for granular contact control.
- MP3D-R: Used to evaluate the model's generalization capabilities in unseen room layouts.
Experimental Results
The method was evaluated against state-of-the-art models including PSI, PLACE, and POSA. Key findings include:
- Physical Plausibility: The model achieved perfect scores (1.0) for both non-collision and contact metrics, indicating a complete absence of clipping and consistent environmental contact.
- Pose Diversity: The variety of generated poses, measured by cluster size, reached 4.69—nearly double the 2.53 achieved by the PSI baseline.
- Efficiency: Unlike previous methods that require "Geo Fitting" optimization steps, this framework produces high-fidelity results directly from the decoder.
Conclusion
This work represents a significant advancement in scene-aware human generation by making contact constraints adaptive to the intended pose. While highly effective, limitations remain in handling extremely complex scene geometries and very fine-grained pose nuances. Future research will focus on expanding pose categories and extending the framework to dynamic motion sequences, moving beyond static snapshots to full interactive movements.
Summary updated