Abstract
TEGLO (Textured EG3D-GLO) is a novel framework that achieves near-perfect 3D reconstruction from a single "in-the-wild" image by disentangling texture from geometry. By mapping input pixels directly into a 2D canonical texture space, it preserves high-frequency details and enables seamless texture editing and transfer without requiring 3D meshes or expensive fine-tuning.
Introduction
While Neural Radiance Fields (NeRFs) have revolutionized 3D scene representation, applying them to "in-the-wild" collections of single-view images (like a dataset of random faces) remains difficult. Current methods face three main hurdles: the loss of high-frequency details due to the low-pass filtering effect of standard MLPs, the entanglement of appearance and geometry, and the heavy computational cost of per-image optimization. TEGLO aims to provide a way to reconstruct high-resolution, 3D-consistent objects from single views while keeping texture and geometry strictly separate for downstream editing.
Methods
The authors propose a two-stage pipeline. In the first stage, a conditional NeRF is trained using a tri-plane representation and Generative Latent Optimization (GLO). This "auto-decoder" setup allows the model to reconstruct images at arbitrary resolutions without being constrained by a discriminator's fixed resolution.
The second stage introduces dense correspondence learning, the core innovation of the framework. The model learns to map 3D surface points from the NeRF onto a 2D canonical coordinate space. A Texture Mapper "unwraps" input pixels onto this map, and Natural Neighbor Interpolation (NNI) is used to fill gaps caused by occlusions, effectively creating a high-resolution texture atlas.
Key Insights
A core innovation of TEGLO is the realization that for reconstruction, the input pixels themselves are the ground truth. By mapping them directly to a texture space, the model bypasses the "spectral bias" of MLPs that usually blurs fine details like skin pores or wrinkles. This creates a universal "atlas" for a given object class, making texture transfer and editing as simple as modifying a 2D image.
Key Results
TEGLO achieves unprecedented fidelity, with a reconstruction PSNR of $\ge$ 74 dB at $1024^2$ resolution, compared to previous state-of-the-art methods that hover around 26-29 dB. In tests on ShapeNet-Cars, it significantly outperformed baselines in multi-view consistency, achieving a PSNR of 67.52 compared to the next best at 30.48. Furthermore, the model demonstrates robust zero-shot editing capabilities, allowing for the addition of "virtual tattoos" or face paint that renders perfectly across all 3D views.
Conclusion
TEGLO represents a significant step forward in high-fidelity 3D reconstruction from single images. By disentangling texture and geometry through a canonical mapping, it enables high-resolution editing and consistent multi-view rendering without the need for complex 3D data. Future work will aim to integrate the pipeline into an end-to-end formulation and improve the handling of environmental lighting and occluded regions.
Summary updated