TL;DR
ArchitectHead is the first framework to enable real-time, continuous Level of Detail (LOD) control for 3D Gaussian Splatting head avatars without requiring retraining. By dynamically resampling a learnable multi-level 2D UV feature field, it can slash the number of Gaussians by up to 93.8% at the lowest detail level, nearly doubling rendering speeds while gracefully maintaining visual quality.
What Problem Does This Paper Address?
3D Gaussian Splatting (3DGS) has revolutionized real-time 3D head avatar rendering, but it suffers from a major rigidity: once trained, the number of Gaussian primitives is locked. In practical applications like VR, gaming, or multi-party video conferencing, rendering dozens of high-fidelity avatars simultaneously destroys performance.
Standard computer graphics solves this using Level of Detail (LOD) meshes, but applying LOD to unstructured, dynamic 3D Gaussian point clouds is incredibly difficult. Existing 3DGS LOD methods either focus on static scenes or only support a few discrete steps, causing jarring visual "popping" during transitions. ArchitectHead bridges this gap by introducing smooth, continuous LOD control for dynamic head avatars, allowing applications to seamlessly balance rendering efficiency and visual quality on the fly.
Methods
Instead of directly manipulating unstructured 3D points, the authors parameterize the Gaussians within a 2D UV feature space. The core innovation is a Multi-Level UV Feature Field—essentially a pyramid of learnable feature maps at varying resolutions (e.g., $256 \times 256$, $128 \times 128$, and $64 \times 64$).
To achieve continuous LOD control at runtime, a Weighted Resampler uses a softmax-based interpolation strategy to blend these feature maps based on a target LOD value ($l \in [0, 1]$). This blended feature map is concatenated with:
- A positionally encoded UV position map (derived from a FLAME mesh prior).
- An expression driving code (compressed via a mapping MLP from FLAME parameters).
- The continuous LOD value itself.
Finally, a lightweight decoder MLP transforms these pixel-wise latent features into standard 3D Gaussian attributes (position offsets, scale, rotation, opacity, and color coefficients) for rendering via standard 3DGS rasterizers.
Training happens in two stages: Stage 1 optimizes the network at the highest resolution to lock in fine details, while Stage 2 randomly samples LOD values to teach the decoder how to gracefully handle lower-resolution feature maps.
Datasets
The authors evaluated their framework on two popular monocular video datasets:
- PointAvatar: Featuring 3 subjects.
- INSTA: Using 9 distinct subjects (one overlapping subject with PointAvatar was omitted).
For each video, the first 90% of the frames were allocated for training, and the remaining 10% were reserved for testing self-reenactment and novel-view synthesis.
Key Insights and Ideas
- Treating Point Clouds Like Images: The core "aha moment" is realizing that while unstructured 3D points cannot be easily scaled, a 2D UV feature map can be resized just like a standard image. This maps the complex problem of point cloud simplification to a well-understood 2D image resampling problem.
- Softmax Blending Prevents Information Loss: Simply downsampling a single high-resolution feature map smooths out critical high-frequency details (like unique rotations and scales of adjacent Gaussians). By maintaining a multi-level pyramid and using a soft-blending mechanism, the network retains gradient flow and transitions smoothly without visual artifacts.
- LOD as a Conditional Input: Explicitly feeding the LOD value into the decoder MLP allows the network to learn a continuous representation space, adapting the shape and size of the generated Gaussians to match the current resolution context.
Key Results
- State-of-the-Art Quality: At the highest LOD (LOD = 0.0), ArchitectHead achieves state-of-the-art performance across self-reenactment and cross-identity tasks, outperforming baselines like FlashAvatar and Gaussian Dejavu in PSNR, SSIM, and LPIPS.
- Massive Pruning with Graceful Degradation: At the lowest LOD (LOD = 1.0), the model drops the Gaussian count from 45,876 down to just 2,829—using only 6.2% of the original primitives. Despite this massive reduction, L1 loss increases by only 7.9%, while PSNR drops by less than 1 dB (-0.97%) and SSIM by a mere 0.6%.
- Double the Speed: This reduction translates directly to performance: rendering speeds nearly double, jumping from 141 FPS to 276 FPS on an NVIDIA RTX 4090 using half-precision (fp16).
Limitations and Future Work
The framework heavily relies on the accuracy of the underlying FLAME head tracker; any tracking errors directly compromise 3D consistency. Furthermore, the model struggles with rare expression modes that only appear under extreme head poses (such as sharp side views), occasionally leading to overfitting and minor artifacts in those specific views.
For future work, the authors plan to scale the framework to handle multi-view video datasets and optimize training efficiency.
Summary updated