Abstract
Neural Radiance Fields (NeRF) have become a cornerstone for generating human avatars, particularly in novel view synthesis. However, existing NeRF-based methods often rely on static features that struggle to adapt to the fine-grained variations caused by dynamic poses and changing lighting. We present D-NeuRA, a human avatar modeling framework that integrates static theme priors with image-driven dynamic features. By employing a novel Multi-Scale Bitemporal Dynamic Feature Fusion module and a Median-Enhanced Channel and Spatial Attention (MECS) mechanism, D-NeuRA achieves high-fidelity rendering with disentangled control over pose, shape, and appearance. Our approach demonstrates superior robustness and generalization in complex dynamic scenes compared to current state-of-the-art methods.
Introduction
Creating a digital avatar that moves naturally and reacts realistically to environmental changes remains a significant challenge in computer vision. While static 3D reconstruction has matured, "dynamic" realism—capturing how skin stretches, muscles flex, and fabric wrinkles during movement—is often lost in translation. Traditional NeRF models typically learn a pose-independent "base" representation. While this facilitates basic customization, it fails to capture pose-dependent details, often resulting in blurred textures or "ghosting" artifacts during complex motions.
Methodology
D-NeuRA addresses these limitations through three primary technical innovations designed to bridge the gap between static stability and dynamic realism.
1. Adaptive Appearance Feature Construction
Rather than relying solely on a pre-learned static latent code, D-NeuRA extracts dynamic features directly from input images. The system maps 3D sampling points to 2D image coordinates, using bilinear sampling to capture local texture and pose information. These dynamic features are then fused with static "theme" features via a Multi-Scale Bitemporal Dynamic Feature Fusion module, ensuring the avatar maintains global consistency while reflecting local, pose-specific details.
2. Two-Stage Pose Enhancement
To refine geometric modeling, the framework employs a two-stage strategy:
- Stage 1: Key Pose Retrieval: The model retrieves and interpolates features from the five most similar poses in the training set to provide a geometric baseline.
- Stage 2: MECS Module: To prevent the "averaging blur" common in interpolation, the Median-Enhanced Channel and Spatial Attention (MECS) module is introduced. By incorporating median pooling—which is inherently robust to outliers—the model preserves sharp geometric cues, such as deep clothing folds, that standard mean or max pooling often smooth out.
3. Disentangled Rendering Pipeline
The framework utilizes separate decoders for SDF (geometry), Albedo (base color), and Shadow (lighting and occlusion). This disentanglement is crucial for customization, allowing users to independently modify an avatar’s body shape, clothing texture, or lighting environment without degrading the underlying model integrity.
Experimental Results
The researchers evaluated D-NeuRA on the ZJU-MoCap (multi-view) and MonoCap (monocular) benchmarks.
Quantitative Performance: D-NeuRA consistently outperformed baselines such as Neural Body (NB) and NECA. On the ZJU-MoCap dataset, it achieved a PSNR of 29.3 and a significantly lower LPIPS (0.062), indicating superior perceptual similarity to ground-truth images.
Qualitative Improvements: The model showed marked improvements in rendering complex areas like facial contours and hand gestures. In "novel pose" synthesis—rendering positions not seen during training—D-NeuRA maintained structural integrity where competing models often suffered from limb collapse or texture blurring. Furthermore, the disentangled pipeline allowed for seamless "reshadowing," where the model realistically updates shadows based on pose changes even under static external lighting.
Conclusion and Future Work
D-NeuRA represents a significant step toward fully customizable, high-fidelity digital humans. By combining the stability of static priors with the detail of image-driven dynamic features, it achieves a level of realism previously difficult to reach in real-time rendering.
Currently, the model's accuracy is linked to the quality of the underlying SMPL (Skinned Multi-Person Linear) pose estimation. Future research will focus on developing "shared representations" to allow the model to generalize across different subjects without requiring per-person optimization, moving the industry closer to a universal human rendering engine.
Summary updated