3DGCQA: A Quality Assessment Database for 3D AI-Generated Contents

Yingjie Zhou, Zicheng Zhang, Farong Wen, Jun Jia, Yanwei Jiang, Xiaohong Liu, Xiongkuo Min, Guangtao Zhai

Abstract

Although 3D generated content (3DGC) offers advantages in reducing production costs and accelerating design timelines, its quality often falls short when compared to 3D professionally generated content (PGC). Common quality issues frequently affect 3DGC, highlighting the importance of timely and effective quality assessment. To address existing gaps, this paper introduces a novel 3DGC quality assessment dataset, 3DGCQA, built using 7 representative Text-to-3D generation methods. Using 50 fixed prompts, the researchers created 313 textured meshes. Subjective quality assessment revealed significant variation across methods, while objective tests exposed limitations in existing algorithms, underscoring the need for specialized 3D quality assessment tools. The dataset is open-sourced at https://github.com/zyj-2000/3DGCQA.

Introduction

While AI can now generate 3D models from simple text prompts, the results are often inconsistent. Unlike professionally generated content (PGC), which maintains high standards of structural integrity and texture alignment, AI-generated content (AIGC) frequently suffers from bizarre geometric distortions and blurry textures.

Comparison between Professional (PGC) and AI-Generated (AIGC) 3D content, highlighting differences in quality, time, and cost.

To advance generative models, precise quality measurement is essential. However, most existing 3D quality assessment (3DQA) tools were designed for scanned real-world objects or compressed point clouds, making them ill-suited for the unique artifacts produced by AI. The 3DGCQA database addresses this by providing a standardized set of AI-generated 3D models paired with human quality scores.

Methodology and Dataset Construction

The researchers utilized 7 representative Text-to-3D methods—including DreamFusion, Magic3D, and HiFA—to generate models based on 50 diverse prompts across 10 categories (e.g., animals, tools, humans). This resulted in 313 textured meshes featuring various artistic styles like "Japanese ink style" or "cyberpunk."

Overview of the 3DGCQA dataset: (a) prompt word cloud, (b) success/failure counts per method, and (c) generation time distributions.

During construction, the team identified 6 common distortion categories unique to 3DGC:

  • Multifaceted Repetition: The "Janus problem," where an object has multiple faces or heads.
  • Depth Error: Incorrect spatial relationships between parts.
  • Roughness: Unintended surface noise on the mesh.
  • Misalignment: Texture and geometry failing to line up.
  • Geometry Loss: Missing components of the intended object.
  • Geometry Redundancy: Extra, floating bits of mesh.

Subjective and Objective Quality Analysis

Subjective scoring was conducted by 40 human evaluators who rated the models on alignment (prompt matching) and overall quality. While most models aligned well with prompts, visual quality scores averaged a mediocre 2.5 out of 5. Notably, human models received the lowest scores due to the high complexity of human textures and the strict geometric symmetry expected by the human eye.

Magic3D emerged as the top-performing generator in terms of visual quality. Furthermore, the choice of the underlying Text-to-Image (T2I) model proved critical; methods utilizing DeepFloyd IF generally yielded higher-quality 3D outputs than those using Stable Diffusion.

In benchmarking objective metrics, the Deep Bilinear Convolutional Neural Network (DBCNN) was the top performer with a Spearman Rank Correlation (SRCC) of 0.5381. Traditional "naturalness" metrics like BRISQUE or NIQE performed poorly (SRCC ~0.20), as they are designed for photos and fail to capture the specific artifacts of AI-generated 3D meshes. This highlights a significant "2D-3D gap," where metrics based on 2D snapshots lose vital geometric information perceived by humans.

Conclusion and Future Work

The 3DGCQA dataset provides a vital foundation for the development of 3D-aware assessment tools. While 313 models is a significant start, larger datasets are needed to train robust deep-learning models. Future research should expand beyond Text-to-3D to include Image-to-3D and Video-to-3D content. Ultimately, the field requires new frameworks that directly analyze 3D data structures, such as meshes or Gaussian Splats, to better capture the nuances of 3D quality.

Summary updated

Abstract

Although 3D generated content (3DGC) offers advantages in reducing production costs and accelerating design timelines, its quality often falls short when compared to 3D professionally generated content. Common quality issues frequently affect 3DGC, highlighting the importance of timely and effective quality assessment. Such evaluations not only ensure a higher standard of 3DGCs for end-users but also provide critical insights for advancing generative technologies. To address existing gaps in this domain, this paper introduces a novel 3DGC quality assessment dataset, 3DGCQA, built using 7 representative Text-to-3D generation methods. During the dataset's construction, 50 fixed prompts are utilized to generate contents across all methods, resulting in the creation of 313 textured meshes that constitute the 3DGCQA dataset. The visualization intuitively reveals the presence of 6 common distortion categories in the generated 3DGCs. To further explore the quality of the 3DGCs, subjective quality assessment is conducted by evaluators, whose ratings reveal significant variation in quality across different generation methods. Additionally, several objective quality assessment algorithms are tested on the 3DGCQA dataset. The results expose limitations in the performance of existing algorithms and underscore the need for developing more specialized quality assessment methods. To provide a valuable resource for future research and development in 3D content generation and quality assessment, the dataset has been open-sourced in https://github.com/zyj-2000/3DGCQA.
BibTeX
@inproceedings{zhou2025dgcqa,
  author = {Yingjie Zhou and Zicheng Zhang and Farong Wen and Jun Jia and Yanwei Jiang and Xiaohong Liu and Xiongkuo Min and Guangtao Zhai},
  title = {{3DGCQA: A Quality Assessment Database for 3D AI-Generated Contents}},
  booktitle = {IEEE International Conference on Acoustics, Speech and Signal Processing},
  year = {2025},
  doi = {10.1109/icassp49660.2025.10889921},
}
Figure