WarpHE4D: Dense 4D Head Map toward Full Head Reconstruction

Jongseob Yun, Yong-Hoon Kwon, Min-Gyu Park, Ju-Mi Kang, Min-Ho Lee, Inho Chang, Ju Hong Yoon, Kuk-Jin YoonICCV 2025

TL;DR

This paper introduces WarpHE4D, a novel framework for reconstructing dense 4D head geometry from single images. By leveraging self-supervised visual features from DINOv2 and representing the head as a "4D head map" (UV coordinates, depth, and confidence), WarpHE4D achieves robust, full-head reconstruction without relying on traditional 3D Morphable Models (3DMMs) or explicit template fitting.

What Problem Does This Paper Address?

Reconstructing detailed 3D human heads from images is a long-standing challenge in computer vision, crucial for applications like animation and virtual avatars. Existing methods often struggle with non-facial regions (like hair) and require complex post-processing to align reconstructed geometry with image pixels. Many approaches rely on 3D Morphable Models (3DMMs), which can oversimplify complex facial structures and limit the accuracy of reconstructions, especially in challenging poses or with significant viewpoint variations. Furthermore, aligning these models to input images often introduces errors that hinder further processing like texture mapping.

Methods

WarpHE4D tackles this by proposing a "dense 4D head map" representation. Instead of fitting a template, it directly regresses a per-pixel 4D vector containing:

  • UV Coordinates: Mapping each pixel to a point on a fixed UV texture map.
  • Nose-Centered Depth: A normalized depth value relative to the nose tip, designed for better consistency across different head sizes and poses.
  • Confidence: Indicating the likelihood of a pixel belonging to the head.

The core of the method is a network architecture that uses two encoders and two decoders. Crucially, it leverages DINOv2, a powerful self-supervised visual representation model.

  • Encoders: One encoder processes the input image, and another processes a fixed UV texture map. Both utilize DINOv2 for feature extraction, capturing robust semantic and geometric information.
  • Decoders: A coarse decoder predicts an initial 4D head map, which is then refined by a fine decoder. This coarse-to-fine approach allows for efficient processing and high-resolution output.

A key innovation is the nose-centered depth normalization, which aims to provide a more stable and comparable depth representation than traditional methods that rely on global min-max scaling. The framework also incorporates a keypoint loss to improve accuracy around critical facial features like the eyes and mouth, which are often noisy in template-based registrations.

Datasets

  • Training: The authors created a large-scale ground truth dataset by rendering 3D head scans (from FaceScape and NPHM) into 4D head maps.
  • Evaluation:
    • FaceScape: Used for evaluating geometric accuracy.
  • Multiface: A benchmark used for evaluating 4D head map accuracy, with registered images and topology-unified tracked meshes.
  • FaceVerse: A newly proposed benchmark created using FaceVerse 3D head scans to ensure subject diversity and evaluate performance across various head poses.

Key Insights and Ideas

  • DINOv2 as a Universal Prior: The paper's central insight is that DINOv2 features are remarkably consistent and robust for head reconstruction, even under pose variations and distortions. This allows them to bypass the need for explicit 3DMM priors.
  • 4D Head Map as a Unified Representation: Representing the head as a dense 4D map (UV, depth, confidence) elegantly solves the correspondence problem and enables seamless integration of multi-view data without explicit alignment.
  • Nose-Centered Depth Normalization: This novel normalization strategy is more robust to variations in head size and hairstyle compared to traditional methods, leading to improved depth accuracy.
  • Beyond Template Fitting: By treating head reconstruction as a per-pixel regression problem, WarpHE4D avoids the limitations of template models, particularly in non-facial regions.

Key Results

  • State-of-the-Art Performance: WarpHE4D outperforms existing methods on FaceScape and FaceVerse benchmarks, especially in challenging pose conditions.
  • Improved Robustness: The method demonstrates significantly improved robustness to pose and viewpoint variations, particularly for rotations beyond 90 degrees.
  • Wider Coverage: Qualitative results show WarpHE4D covers a larger portion of the head (including non-facial regions) compared to template-based methods, while maintaining lower UV and 3D errors.
  • Efficiency: The framework can achieve 5 FPS (fp32) on an ADA5000 GPU after training, with potential for further speedup by precomputing UV features.

Limitations and Future Work

  • Hair Representation: The authors acknowledge that representing hair as a mesh remains a challenge, and performance might degrade for diverse hair types.
  • Future Work: They aim to explore new representations for non-facial regions and potentially improve hair reconstruction.

Summary updated

BibTeX
@inproceedings{yun2025warphe,
  author = {Jongseob Yun and Yong-Hoon Kwon and Min-Gyu Park and Ju-Mi Kang and Min-Ho Lee and Inho Chang and Ju Hong Yoon and Kuk-Jin Yoon},
  title = {{WarpHE4D: Dense 4D Head Map toward Full Head Reconstruction}},
  booktitle = {IEEE/CVF International Conference on Computer Vision},
  year = {2025},
}
Figure