Complexity-guided Regularization for Generalizable Human Gaussian Splatting

Gun Ryu, Gi-Mun Um, Won-Sik Cheong, Wonjun KimarXiv 2026 · arXiv

TL;DR

This paper introduces a complexity-guided approach to generalizable human Gaussian splatting, addressing the common issue of blurry details in complex regions like fingers and textured clothing. By leveraging a "complexity prior" based on local entropy, the authors intelligently regularize Gaussian density and focus training supervision on intricate areas, achieving state-of-the-art reconstruction quality while maintaining real-time inference speeds.

What Problem Does This Paper Address?

3D Gaussian Splatting (3DGS) has revolutionized real-time rendering, but applying it to "generalizable" human reconstruction—where a model predicts 3D attributes for a new person in a single forward pass without per-subject optimization—remains difficult.

Existing feed-forward methods typically place Gaussians based on fixed image grids or mesh vertices. This uniform approach ignores the fact that some parts of a human (like a plain t-shirt) are visually simple, while others (like fingers or intricate lace) are highly complex. Without a mechanism to increase Gaussian density in these "high-frequency" regions, the resulting renders often look blurry or lack fine-grained structural detail.

Methods

The authors propose two main innovations to solve the density problem:

  1. Complexity-guided Regularization: The core idea is to allocate more Gaussians where the signal is most complex. They define a complexity prior for each vertex of a human mesh by calculating the normalized Shannon entropy of local geometric (surface normals) and textural (color) variations.
  2. Geometric Complexity: Measures how much surface normals vary in a local neighborhood.
  3. Textural Complexity: Measures local color deviations. They then use a pairwise ranking loss ($L_{comp}$) during training. If point A is more complex than point B according to the prior, the model is penalized if it doesn't place more Gaussians around point A.
Overview of the complexity-guided regularization process.
  1. Complexity-aware Patch-based Supervision: Standard training often supervises the model using the entire image. However, complex regions like fingers occupy very few pixels, so their error signal gets "drowned out" by simpler regions like the torso. The authors introduce a strategy that samples small local patches for supervision, with the sampling probability proportional to the complexity prior. This forces the model to "pay more attention" to difficult details during training.

The overall architecture uses a multi-view transformer encoder-decoder to regress Gaussian attributes (position, scale, rotation, opacity, and color) from input images and Plücker rays.

Overall architecture of the proposed method.

Datasets

The model was trained and evaluated on three high-quality 3D human scan datasets:

  • THuman2.0: A standard benchmark for human reconstruction.
  • 2K2K: High-resolution human scans.
  • X-Humans: Expressive human avatars.

To test real-world generalization, they also evaluated on DNA-Rendering and ZJU-MoCap without any fine-tuning.

Key Insights and Ideas

  • Entropy as a Proxy for Density: Using Shannon entropy to quantify "complexity" provides a mathematically grounded way to tell the model where it needs more "representational power" (i.e., more Gaussians).
  • Ranking over Regression: Instead of forcing the model to hit a specific density number, the authors use a ranking loss. This encourages the relative distribution of Gaussians to match the complexity of the subject, which is more robust than absolute density regression.
  • The "Drowning Signal" Problem: A key insight is that global image losses fail for fine details not just because of the model architecture, but because the loss function itself is dominated by large, simple areas. Patch-based sampling effectively "re-balances" the training data.

Key Results

The proposed method sets a new state-of-the-art for generalizable human Gaussian splatting:

  • Quantitative: On the THuman2.0 dataset, it achieved a 32.42 dB PSNR, outperforming the previous best method (GHGS-MVSC) by 1.49 dB.
  • Qualitative: As seen in Figure 1, the method produces significantly sharper results for fingers and clothing textures compared to existing baselines like GPS-Gaussian and RoGSplat.
  • Efficiency: Despite the improved quality, the model runs at ~34 FPS on an NVIDIA RTX 4090, making it suitable for real-time applications.
Figure 1: Distributions of Gaussian centers and the corresponding rendering results. (a) Novel view image. (b) GPS-Gaussian [10]. (c) RoGSplat [15]. (d) Proposed method.

Limitations and Future Work

  • Static Reconstruction: The current framework focuses on static images. Applying it to video would require per-frame reconstruction, which might lead to temporal flickering. The authors suggest integrating parametric models like SMPL to handle motion.
  • Mesh Dependency: The complexity prior currently requires a clothed human mesh during the training phase. While this is available in research datasets, it's harder to get for "in-the-wild" training. Future work could involve computing complexity directly from multi-view images.
  • Identity Preservation: While the model performs well, the authors acknowledge that maintaining perfect identity (facial features) is a persistent challenge in generalizable reconstruction.

Summary updated

Abstract

Recent studies on generalizable human Gaussian splatting have achieved the notable progress in synthesizing novel views of unseen human subjects from multi-view images. Despite this progress, existing methods still struggle to represent fine-grained details of human subjects, which inherently involve high geometric and textural complexity. This is because previous methods do not impose a constraint on how densely Gaussians are placed, which often leads to blurry rendering results in complex regions. To address this problem, we propose a complexity-guided regularization scheme. The key idea is to leverage a complexity prior computed via the local entropy in geometric and textural characteristics of a clothed human. This effectively prevents Gaussians from being coarsely placed in complex regions, which facilitates the faithful reconstruction of intricate structures. Furthermore, we sample local patches predominantly from complex regions according to this complexity prior, which allows the model to be sufficiently supervised by local details. Experimental results on benchmark datasets demonstrate that our method delivers state-of-the-art performance while maintaining the real-time inference speed in generalizable human Gaussian splatting. The code and model are publicly available at https://github.com/DCVL-3D/CGR-GS_release.
BibTeX
@misc{ryu2026complexity,
  author = {Gun Ryu and Gi-Mun Um and Won-Sik Cheong and Wonjun Kim},
  title = {{Complexity-guided Regularization for Generalizable Human Gaussian Splatting}},
  year = {2026},
  eprint = {2610.35152},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2610.35152},
}
Figure