Perceptual Quality Assessment for Digital Human Heads

Zicheng Zhang, Yingjie Zhou, Wei Sun, Xiongkuo Min, Yuzhe Wu, Guangtao Zhai

TL;DR

Researchers have introduced DHHQA, the first large-scale database specifically designed for assessing the visual quality of 3D scanned digital human heads. Alongside this dataset, they proposed a new full-reference objective metric that leverages a hierarchical Swin Transformer and multi-head attention to significantly outperform existing state-of-the-art quality assessment methods.

What Problem Does This Paper Address?

While the "metaverse" and high-fidelity digital humans are becoming mainstream in gaming and VR/AR, our ability to objectively measure their visual quality has lagged behind. Most existing 3D Quality Assessment (3D-QA) research focuses on general objects (like statues or fruit) or point clouds.

Human heads are a unique challenge: we are evolutionarily tuned to notice even tiny artifacts in faces (the "uncanny valley" effect). Existing metrics developed for natural images or general 3D meshes often fail to capture the specific perceptual nuances of digital human heads. Furthermore, there hasn't been a large-scale, publicly available dataset of distorted 3D human heads with human-provided quality scores to train better models.

Methods

The authors tackle this with a two-pronged approach: a new dataset and a new objective metric.

1. The DHHQA Database

They scanned 255 human subjects and selected 55 high-quality reference models. To create a robust benchmark, they applied seven types of common distortions (like surface simplification, texture compression, and geometry noise) at four different levels, resulting in 1,540 distorted models. They then conducted a subjective experiment with 40 participants to collect Mean Opinion Scores (MOS), providing a "ground truth" for human perception.

2. The Objective Metric

Instead of analyzing the complex 3D mesh directly (which is computationally expensive), the authors used a projection-based method. They rendered the 3D heads into 2D images and analyzed those.

Fig. 5. The framework of the proposed method.

Key technical components include:

  • Hierarchical Feature Extraction: They used a pretrained Swin Transformer tiny (ST-t). By extracting features from multiple stages of the network, the model captures both low-level details (like noise and texture) and high-level semantic information.
  • Multi-Head Attention Fusion: To compare the reference head with the distorted one, they used a multi-head attention module. This "actively relates" the features of the two models, allowing the network to learn exactly where and how the quality has degraded.
  • Regression: Finally, a simple fully-connected network maps these fused features to a predicted quality score.

Datasets

The primary contribution is the DHHQA (Digital Human Head Quality Assessment) database. The authors compared it against several existing 3D-QA benchmarks:

Comparison of DHHQA with other 3D-QA databases.

As shown in the table, DHHQA is uniquely focused on scanned real human heads and provides a significant number of subject-rated models compared to earlier mesh-based datasets like CMDM.

Fig. 4. The histogram distribution of the MOSs in the pro- posed database.

Key Insights and Ideas

  • Projections are Practical: While model-based metrics (analyzing vertices directly) are viewpoint-invariant, they struggle with the massive density of high-quality 3D scans. Projection-based methods allow researchers to leverage powerful, pretrained 2D vision transformers.
  • Hierarchical Perception: The success of the multi-stage Swin Transformer approach suggests that human quality judgment isn't just about pixel-level noise; it's a combination of fine-grained texture loss and broader semantic distortion.
  • Domain Gap: Learning-based methods (like LPIPS and the proposed model) significantly beat handcrafted metrics (like PSNR or SSIM). This is likely because digital renders have different statistical properties than natural photos, and models need to be specifically trained on rendered data to "understand" its quality.

Key Results

The proposed method achieved state-of-the-art performance across all standard correlation metrics, including Spearman Rank Correlation Coefficient (SRCC) and Pearson Linear Correlation Coefficient (PLCC).

Table 2. Performance comparison on the DHHQA database. Best in bold.

With an SRCC of 0.9286, the proposed model notably outperformed the next best competitor, LPIPS (0.8935). It also vastly outperformed traditional point-cloud metrics (like MSE-p2point), proving that specialized tools are necessary for textured meshes.

Limitations and Future Work

  • Viewpoint Dependency: The current objective metric only uses front projections for feature extraction. While the subjective tests used both front and side views, the model might miss distortions only visible from the profile or top.
  • Reference Requirement: This is a "Full-Reference" (FR) metric, meaning you need the original, perfect 3D scan to evaluate a distorted one. In many real-world scenarios (like streaming), a "No-Reference" (NR) metric would be more useful.
  • Scale: While 55 reference heads is a great start for this niche, expanding the diversity of the source models (different ethnicities, lighting conditions, etc.) would further improve the robustness of the findings.

Summary updated

Abstract

Digital humans are attracting more and more research interest during the last decade, the generation, representation, rendering, and animation of which have been put into large amounts of effort. However, the quality assessment of digital humans has fallen behind. Therefore, to tackle the challenge of digital human quality assessment issues, we propose the first large-scale quality assessment database for three-dimensional (3D) scanned digital human heads (DHHs). The constructed database consists of 55 reference DHHs and 1,540 distorted DHHs along with the subjective perceptual ratings. Then, a simple yet effective full-reference (FR) projection-based method is proposed to evaluate the visual quality of DHHs. The pretrained Swin Transformer tiny is employed for hierarchical feature extraction and the multi-head attention module is utilized for feature fusion. The experimental results reveal that the proposed method exhibits state-of-the-art performance among the mainstream FR metrics. The database is released at https://github.com/zzc-1998/DHHQA.
BibTeX
@inproceedings{zhang2023perceptual,
  author = {Zicheng Zhang and Yingjie Zhou and Wei Sun and Xiongkuo Min and Yuzhe Wu and Guangtao Zhai},
  title = {{Perceptual Quality Assessment for Digital Human Heads}},
  booktitle = {IEEE International Conference on Acoustics, Speech and Signal Processing},
  year = {2023},
  doi = {10.1109/icassp49357.2023.10095347},
}
Figure