Generated 3D objects are judged mostly by people looking at pictures of them. Raters watch a rendered fly-around video and score how good the object looks, and those scores become benchmarks, training targets, and evidence that one generator beats another. A game engine or physics simulator does not use the picture. It uses the triangle mesh underneath, and a mesh can look flawless while being unusable: its surface torn into unwelded pieces, faces crossing through each other, hidden fragments floating inside. This note treats human quality ratings as self-reports, in the survey-methodology sense: subjective judgments that can drift in systematic, predictable ways from a physical ground truth. Health research handles this by checking such reports against passive sensor traces. We propose the same move for 3D evaluation. We will inject controlled mesh defects that leave the rendered images pixel-identical, swap textures while holding geometry fixed, and run every asset through a physics simulator to obtain an objective outcome. Together these let us measure how much of a human "structural realism" score actually tracks structure.
The broader problem is the gap between how 3D models are generated and what production pipelines need. Neural fields, point clouds and Gaussian splats (small fuzzy blobs of color used as rendering primitives) are easy to optimize but have no explicit surface connectivity, while engines and simulators need clean triangle meshes. Several lines of work try to close this gap. If the human judgments used to decide whether they succeeded cannot see connectivity, that evaluation loop is broken in a way nobody has measured.
Core Claim
We claim that human 3D quality ratings contain a halo effect that can be measured and corrected: texture fidelity and match to the text prompt inflate scores on the dimension meant to capture geometric and structural integrity. We also claim that human ratings are close to blind to connectivity defects that do not show up in renders. The first claim is testable by swapping textures on fixed geometry. The second is testable by comparing ratings of mesh pairs whose renders are identical but whose topology differs. If both hold, the right fix is not "more raters." The fix is to split the human score into the part that tracks appearance and the part that tracks structure, and to anchor the structural part in objective physical checks. This is the same role wearables play against mood diaries in Assessing Real-Time Moderation for Developing Adaptive Mobile Health Interventions for Medical Interns.
The closest existing work is Multi-Dimensional Quality Assessment for Text-to-3D Assets. It collected 49,419 human ratings of 969 generated assets on Quality, Authenticity and Correspondence, and trained a predictor, T23DAQA, from 360-degree projection videos. That paper motivates ours: it reports that raters score semantic Correspondence higher than geometric Authenticity. It also constrains ours, because its ratings and model live entirely in rendered 2D views and never examine manifoldness or connectivity. It treats human scores as ground truth. We treat them as the object of study.
The human studies in MeshGPT raise a similar question. Its authors report that 73% of raters preferred their triangulations as "cleaner" than GET3D's dense extractions. That forced-choice judgment was never checked against an objective measure of topology, and it may partly reflect the visual appeal of low face counts.
Research Program
Asset pool. We build a pool of explicit meshes from two sources. The first is assets from AIGC-T23DAQA, converted to meshes with Marching Cubes, the standard method for turning a density or distance field into triangles. The second is compact generated meshes in the ShapeNet categories MeshGPT reports on. Marching Cubes on neural fields is known to produce fragmented, non-manifold output (TextMesh), so the pool will naturally span clean and defective topology.
Render-invariant defects. For each clean base mesh we generate variants whose defects cannot be seen in renders. These include unwelded duplicate vertices along seams, fully occluded interior floaters, self-intersections sealed inside the volume, and non-manifold edges hidden in concavities. We verify pixel identity under the rating renderer.
Visible-defect controls. We add matched variants in which the same defect classes are visible, for example open holes, as a sanity check that raters can detect defects when they can see them.
Texture swaps. A third family keeps geometry fixed and swaps high-fidelity and degraded textures. This isolates the halo effect on the Authenticity dimension.
Objective outcome. Every asset gets a physical score that plays the role of a sensor trace. We pass each mesh to an unmodified physics engine under a fixed protocol of gravity drop, soft-body compression and cloth draping where applicable. This follows GASP in treating the engine as a black box that consumes raw triangles. We record simulation failure, meaning explosion, interpenetration or tearing. We also record static checks: connected components, manifoldness and self-intersection count.
Rating conditions. We collect ratings using the three-dimension protocol of Fu et al. under several conditions. The first is the original setting: shaded fly-around videos, non-expert raters. The second shows shaded renders plus a wireframe overlay. The third adds a short simulation preview clip. The fourth uses 3D-artist raters. Crossed with the conditions is a disclosure manipulation, in which assets are labeled "AI-generated" or unlabeled, to probe demand effects of the kind documented by Tag2Risk in clinical questionnaires.
Measurement model. The paired design lets us fit a per-rater, per-condition measurement model:
$$r_{ij} = \mu_j + \alpha_j\, A_i + \beta_j\, T_i + \epsilon_{ij}.$$
Here $r_{ij}$ is rater $j$'s Authenticity score for asset $i$; $A_i$ is an appearance factor, set by the texture-swap manipulation; $T_i$ is an objective topology/simulation factor, set by defect injection and simulation outcome; $\mu_j$ is the rater's leniency; and $\epsilon_{ij}$ is noise. Because the variants are controlled pairs, $\alpha_j$ and $\beta_j$ are identified from within-asset contrasts rather than from confounded observational variation. The halo hypothesis predicts a large $\alpha$ and a near-zero $\beta$ for render-invariant defects.
Corrected assessor. The deliverable is a corrected Authenticity score and a hybrid assessor. The assessor keeps human judgment where humans are informative (appearance, prompt match) and replaces or reweights the structural component with the objective checks, weighted by the estimated $\beta$.
Baselines
Naive reference. The raw mean human opinion score under the original render-only protocol. This is what current benchmarks use. We beat it if our corrected score correlates better with simulation success on held-out assets, measured by rank correlation and by the ability to separate simulation-failing from simulation-passing meshes (area under the ROC curve).
Strongest learned competitor. T23DAQA from Multi-Dimensional Quality Assessment for Text-to-3D Assets. Its authors report Spearman rank correlation (SRCC) of 0.7853 for Quality, 0.6728 for Authenticity and 0.7000 for Correspondence against human scores on AIGC-T23DAQA, outperforming 34 baseline metrics. It is the strongest competitor because it is the best existing predictor of exactly the human judgments we are questioning. For fairness, we retrain it on the same asset pool with the same splits and the same hyperparameter-search budget as our assessor. The key test is on render-invariant pairs, where any purely render-based model is forced to give identical scores. Beating it means higher correlation with simulation outcomes while staying within a small margin (about 0.02 SRCC) of its reported Quality and Correspondence correlations with humans. No reported result was found on Lacuna for T23DAQA or any assessor against physical simulation outcomes, so that axis has no prior number.
Geometry-only checker. Static topology metrics plus simulation, with no human input at all. This is the upper bound on structure prediction and the lower bound on prompt-match and appearance prediction.
Ablations. A T23DAQA-style model with a mesh-graph branch added, which tests whether architecture alone recovers structure without the measurement model. Our assessor with the measurement-model correction removed, which isolates the key ingredient.
Why not just…?
Why not just drop humans and score topology automatically? This is a real threat, and the proposal is built to settle it rather than argue around it. If static checks and simulation predict everything practitioners care about in "Authenticity," the right conclusion is that this dimension should be computed, not rated. We would report that as the main finding. We expect humans to stay necessary for prompt match and for visible aesthetic plausibility: Fu et al. report 0.7000 SRCC for Correspondence from a vision-language branch, well short of perfect. The geometry-only baseline measures the exact boundary.
Why not just show raters wireframes or hire experts? It may partly work, which is why both appear as rating conditions and not as a strawman. Render-invariant defects such as unwelded seams and sealed interior intersections do not appear in wireframes of the visible surface. Our prediction is therefore that experts and wireframes raise $\beta$ for visible defects but not for invisible ones. If experts do detect invisible defects, the practical recommendation becomes a cheaper protocol change, and the measurement model still quantifies how much it buys.
Why not just extract with Marching Cubes and repair afterward, so topology never needs evaluating? Post-hoc extraction is exactly what produces fragmented, over-tessellated meshes (TextMesh; MeshGPT's comparison of about 228 versus 27,000 faces). Geometry-only simplification also degrades appearance (Appearance-Driven Automatic 3D Model Simplification). Repair does not remove the need to measure whether evaluation sees the defects that repair is supposed to fix.
Proposed Experiments
The first decisive check is the render-invariant pair test. For each pair of clean mesh and invisibly defective mesh, compute the difference in human Authenticity score and in simulation success. If raters are topology-blind, the score difference will be statistically indistinguishable from zero while the simulation failure rate differs sharply. We will estimate the required sample size in a pilot and pre-register the equivalence margin.
The second check is the texture-swap test on fixed geometry. A significant Authenticity shift under texture swap is direct evidence of halo contamination of the structural dimension. It would also give a mechanism for the Correspondence-over-Authenticity pattern that Fu et al. observed.
The third check is predictive. On held-out assets and held-out generators, compare the naive score, retrained T23DAQA, the geometry-only checker, and the corrected hybrid on two things: correlation with simulation success, and agreement with human Correspondence and Quality.
The fourth check replicates MeshGPT's "cleaner triangulation" forced-choice question on pairs where face count and topological validity are decorrelated by design. This tests whether the reported 73% preference reflects connectivity or face count.
The proposal fails informatively if raters turn out to be sensitive to invisible defects. That outcome would vindicate existing 3D benchmarks. It succeeds if it shows, with controlled pairs, that current human scores reward assets that look right and break in simulation, and if it supplies a corrected score that generators aimed at simulation-ready meshes can safely optimize.
References
- Assessing Real-Time Moderation for Developing Adaptive Mobile Health Interventions for Medical Interns: Micro-Randomized Trial
- Multi-Dimensional Quality Assessment for Text-to-3D Assets: Dataset and Model
- MeshGPT: Generating Triangle Meshes with Decoder-Only Transformers
- TextMesh: Generation of Realistic 3D Meshes From Text Prompts
- GASP: Gaussian Splatting for physics-based simulations
- Tag2Risk: Harnessing Social Music Tags for Characterizing Depression Risk
- Appearance-Driven Automatic 3D Model Simplification