Multi-Dimensional Quality Assessment for Text-to-3D Assets: Dataset and Model

Kang Fu, Huiyu Duan, Zicheng Zhang, Xiaohong Liu, Xiongkuo Min, Jia Wang, Guangtao Zhai

Abstract

The rapid advancement of diffusion models has enabled the generation of complex 3D assets from text prompts. However, evaluating these assets remains a challenge due to unique AI-generated artifacts and the lack of alignment between traditional 3D metrics and human perception. This paper introduces the AIGC-T23DAQA database, the first large-scale dataset for text-to-3D quality assessment, and proposes a multi-dimensional objective evaluation model, T23DAQA. By leveraging a projection-based approach and multi-modal feature extraction, the model achieves state-of-the-art performance in predicting human scores for quality, authenticity, and text-asset correspondence.

Introduction: The Challenge of AI-Generated 3D Evaluation

Evaluating AI-generated 3D assets is inherently difficult. Unlike traditional 3D models created via professional software or scanning, AI-generated assets—often represented as Neural Radiance Fields (NeRFs)—suffer from novel geometric distortions such as "floaters" and the "Janus problem." Furthermore, traditional metrics designed for meshes or point clouds are incompatible with implicit representations without lossy conversion. Most importantly, existing metrics lack the context to determine if an asset actually matches the user's intent. This research addresses these gaps by aligning automated assessment with human perception across three distinct dimensions: quality, authenticity, and correspondence.

Methodology: The T23DAQA Model

To bridge the gap between human judgment and automated scoring, the authors propose the T23DAQA (Text-to-3D Asset Quality Assessment) model. Recognizing that NeRFs are computationally intensive to render in real-time, the model adopts a projection-based approach, utilizing 360-degree fly-around videos as the primary input.

The T23DAQA model architecture utilizes three parallel branches for shape, texture, and text-alignment evaluation.

The model architecture consists of three specialized branches:

  1. Shape Feature Encoder: Employs a 3D Video Swin Transformer (Swin3D-s) to detect geometric anomalies and floating artifacts within the projection sequence.
  2. Texture Feature Encoder: Uses a 2D Swin Transformer on key frames (front and back) to assess material properties and visual fidelity.
  3. Text-Image Alignment Encoder: Leverages a pre-trained CLIP model to evaluate the correspondence between the 3D asset's visual projection and the original text prompt.

These features are integrated via a Multi-Layer Perceptron (MLP) to generate three distinct scores aligned with human subjective ratings.

The AIGC-T23DAQA Database

A central contribution of this work is the AIGC-T23DAQA database, the largest dataset of its kind. It comprises 969 validated 3D assets generated from 170 diverse prompts using six popular models, including DreamFusion and Magic3D.

The AIGC-T23DAQA database includes diverse prompts and categories for comprehensive quality assessment.

The dataset includes 49,419 subjective human ratings. These ratings provide a ground truth for three critical dimensions:

  • Quality: Visual attributes such as texture and color.
  • Authenticity: Geometric realism and structural integrity.
  • Correspondence: How well the asset aligns with the semantic meaning of the prompt.

Key Insights and Results

Analysis of the dataset reveals that current text-to-3D models prioritize text matching over 3D realism, scoring higher on "Correspondence" than on "Quality" or "Authenticity." Additionally, prompt length significantly impacts generation: while short and very long prompts yield high scores, medium-length prompts often confuse models, leading to a drop in all quality dimensions.

Quantitatively, the T23DAQA model achieves state-of-the-art performance, outperforming 34 baseline metrics across image, video, and 3D domains. It achieved a Spearman Rank Correlation Coefficient (SRCC) of 0.7853 for Quality, 0.6728 for Authenticity, and 0.7000 for Correspondence, proving strong alignment with human subjective ratings. Ablation studies highlighted the CLIP-based text-image alignment module as the most critical feature for predicting human preference.

Conclusion and Future Directions

While the current model relies on 2D projections, it provides a highly effective proxy for 3D quality assessment. The T23DAQA framework holds immense potential as an automated filter for commercial 3D generation platforms and could potentially be used as a loss function during the training of future models to directly optimize for human perceptual preference.

Summary updated

Abstract

Recent advancements in text-to-image (T2I) generation have spurred the development of text-to-3D asset (T23DA) generation, leveraging pretrained 2D text-to-image diffusion models for text-to-3D asset synthesis. Despite the growing popularity of text-to-3D asset generation, its evaluation has not been well considered and studied. However, given the significant quality discrepancies among various text-to-3D assets, there is a pressing need for quality assessment models aligned with human subjective judgments. To tackle this challenge, we conduct a comprehensive study to explore the T23DA quality assessment (T23DAQA) problem in this work from both subjective and objective perspectives. Given the absence of corresponding databases, we first establish the largest text-to-3D asset quality assessment database to date, termed the AIGC-T23DAQA database. This database encompasses 969 validated 3D assets generated from 170 prompts via 6 popular text-to-3D asset generation models, and corresponding subjective quality ratings for these assets from the perspectives of quality, authenticity, and text-asset correspondence, respectively. Subsequently, we establish a comprehensive benchmark based on the AIGC-T23DAQA database, and devise an effective T23DAQA model to evaluate the generated 3D assets from the aforementioned three perspectives, respectively.
BibTeX
@article{fu2025multi,
  author = {Kang Fu and Huiyu Duan and Zicheng Zhang and Xiaohong Liu and Xiongkuo Min and Jia Wang and Guangtao Zhai},
  title = {{Multi-Dimensional Quality Assessment for Text-to-3D Assets: Dataset and Model}},
  journal = {IEEE Trans. Multimedia},
  year = {2025},
  doi = {10.1109/tmm.2025.3604905},
}
Figure