ArchitectHead: Continuous Level of Detail Control for 3D Gaussian Head Avatars

Peizhi Yan, Rabab Ward, Qiang Tang, Shan Du

TL;DR

ArchitectHead is the first framework to enable real-time, continuous Level of Detail (LOD) control for 3D Gaussian Splatting head avatars without requiring retraining. By dynamically resampling a learnable multi-level 2D UV feature field, it can slash the number of Gaussians by up to 93.8% at the lowest detail level, nearly doubling rendering speeds while gracefully maintaining visual quality.

What Problem Does This Paper Address?

3D Gaussian Splatting (3DGS) has revolutionized real-time 3D head avatar rendering, but it suffers from a major rigidity: once trained, the number of Gaussian primitives is locked. In practical applications like VR, gaming, or multi-party video conferencing, rendering dozens of high-fidelity avatars simultaneously destroys performance.

Standard computer graphics solves this using Level of Detail (LOD) meshes, but applying LOD to unstructured, dynamic 3D Gaussian point clouds is incredibly difficult. Existing 3DGS LOD methods either focus on static scenes or only support a few discrete steps, causing jarring visual "popping" during transitions. ArchitectHead bridges this gap by introducing smooth, continuous LOD control for dynamic head avatars, allowing applications to seamlessly balance rendering efficiency and visual quality on the fly.

Methods

Instead of directly manipulating unstructured 3D points, the authors parameterize the Gaussians within a 2D UV feature space. The core innovation is a Multi-Level UV Feature Field—essentially a pyramid of learnable feature maps at varying resolutions (e.g., $256 \times 256$, $128 \times 128$, and $64 \times 64$).

Figure 2. Pipeline of ArchitectHead. We propose a 3D Gaussian head avatar creation method with continuous level of detail (LOD) control. Starting from shape and expression codes, we use the FLAME head model to generate the 3D mesh geometry, which is rasterized into a UV position map at the desired resolution. A multi-level UV feature field is introduced to learn local latent features, from which our weighted resampler extracts a UV feature map of the target resolution. This map is concatenated with the UV position map, the desired LOD value, and a driving code obtained from expression and pose codes via an MLP network M. The resulting pixel-wise latent features are decoded by an MLP-based decoder into 3D Gaussian attributes, which are rendered using 3D Gaussian Splatting (3DGS).

To achieve continuous LOD control at runtime, a Weighted Resampler uses a softmax-based interpolation strategy to blend these feature maps based on a target LOD value ($l \in [0, 1]$). This blended feature map is concatenated with:

  1. A positionally encoded UV position map (derived from a FLAME mesh prior).
  2. An expression driving code (compressed via a mapping MLP from FLAME parameters).
  3. The continuous LOD value itself.

Finally, a lightweight decoder MLP transforms these pixel-wise latent features into standard 3D Gaussian attributes (position offsets, scale, rotation, opacity, and color coefficients) for rendering via standard 3DGS rasterizers.

Training happens in two stages: Stage 1 optimizes the network at the highest resolution to lock in fine details, while Stage 2 randomly samples LOD values to teach the decoder how to gracefully handle lower-resolution feature maps.

Datasets

The authors evaluated their framework on two popular monocular video datasets:

  • PointAvatar: Featuring 3 subjects.
  • INSTA: Using 9 distinct subjects (one overlapping subject with PointAvatar was omitted).

For each video, the first 90% of the frames were allocated for training, and the remaining 10% were reserved for testing self-reenactment and novel-view synthesis.

Key Insights and Ideas

  • Treating Point Clouds Like Images: The core "aha moment" is realizing that while unstructured 3D points cannot be easily scaled, a 2D UV feature map can be resized just like a standard image. This maps the complex problem of point cloud simplification to a well-understood 2D image resampling problem.
  • Softmax Blending Prevents Information Loss: Simply downsampling a single high-resolution feature map smooths out critical high-frequency details (like unique rotations and scales of adjacent Gaussians). By maintaining a multi-level pyramid and using a soft-blending mechanism, the network retains gradient flow and transitions smoothly without visual artifacts.
  • LOD as a Conditional Input: Explicitly feeding the LOD value into the decoder MLP allows the network to learn a continuous representation space, adapting the shape and size of the generated Gaussians to match the current resolution context.

Key Results

  • State-of-the-Art Quality: At the highest LOD (LOD = 0.0), ArchitectHead achieves state-of-the-art performance across self-reenactment and cross-identity tasks, outperforming baselines like FlashAvatar and Gaussian Dejavu in PSNR, SSIM, and LPIPS.
  • Massive Pruning with Graceful Degradation: At the lowest LOD (LOD = 1.0), the model drops the Gaussian count from 45,876 down to just 2,829—using only 6.2% of the original primitives. Despite this massive reduction, L1 loss increases by only 7.9%, while PSNR drops by less than 1 dB (-0.97%) and SSIM by a mere 0.6%.
  • Double the Speed: This reduction translates directly to performance: rendering speeds nearly double, jumping from 141 FPS to 276 FPS on an NVIDIA RTX 4090 using half-precision (fp16).
Figure 1. ArchitectHead supports continuous LOD control ranging from 0 (highest) to 1.0 (lowest). This figure shows the rendered image quality under different LOD settings. For each case, we provide a zoom-in view of selected regions. We recommend viewing the figure on a digital device with zoom for better inspection. The grey dots indicate the positions of the 3D Gaussian points. The test identity is taken from the NeRFace dataset [7]. Red arrow indicates visible artifacts (sparse Gaussians) in the lowest LOD.

Limitations and Future Work

The framework heavily relies on the accuracy of the underlying FLAME head tracker; any tracking errors directly compromise 3D consistency. Furthermore, the model struggles with rare expression modes that only appear under extreme head poses (such as sharp side views), occasionally leading to overfitting and minor artifacts in those specific views.

For future work, the authors plan to scale the framework to handle multi-view video datasets and optimize training efficiency.

Summary updated

Abstract

3D Gaussian Splatting (3DGS) has enabled photorealistic and real-time rendering of 3D head avatars. Existing 3DGS-based avatars typically rely on tens of thousands of 3D Gaussian points (Gaussians), with the number of Gaussians fixed after training. However, many practical applications require adjustable levels of detail (LOD) to balance rendering efficiency and visual quality. In this work, we propose "ArchitectHead", the first framework for creating 3D Gaussian head avatars that support continuous control over LOD. Our key idea is to parameterize the Gaussians in a 2D UV feature space and propose a UV feature field composed of multi-level learnable feature maps to encode their latent features. A lightweight neural network-based decoder then transforms these latent features into 3D Gaussian attributes for rendering. ArchitectHead controls the number of Gaussians by dynamically resampling feature maps from the UV feature field at the desired resolutions. This method enables efficient and continuous control of LOD without retraining. Experimental results show that ArchitectHead achieves state-of-the-art (SOTA) quality in self and cross-identity reenactment tasks at the highest LOD, while maintaining near SOTA performance at lower LODs. At the lowest LOD, our method uses only 6.2\% of the Gaussians while the quality degrades moderately (L1 Loss +7.9\%, PSNR --0.97\%, SSIM --0.6\%, LPIPS Loss +24.1\%), and the rendering speed nearly doubles.
BibTeX
@inproceedings{yan2026architecthead,
  author = {Peizhi Yan and Rabab Ward and Qiang Tang and Shan Du},
  title = {{ArchitectHead: Continuous Level of Detail Control for 3D Gaussian Head Avatars}},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision},
  year = {2026},
}
Figure