Traditional 3D quality assessment treated point clouds and meshes purely as geometric entities. Metrics like point-to-point root-mean-square error or Hausdorff distance measured how far a reconstructed vertex or point deviated from its original spatial position. While mathematically convenient, this approach fails to capture how humans actually experience 3D content.
Humans do not perceive 3D models as raw wireframes or uncolored point clouds; we see fully rendered surfaces where shape and appearance are inextricably linked. To build objective metrics that truly align with human judgment, modern research has shifted toward multimodal quality assessment—explicitly modeling the complex, non-linear interactions between 3D geometry and 2D surface textures.
Beyond Geometric Distance
When an asset undergoes compression or transmission loss, geometric-only metrics often produce scores that diverge sharply from human subjective ratings. As Xinju Wu et al. demonstrate in their subjective study of compressed point clouds, human observers are frequently far more sensitive to color bleeding and texture blurring than to minor shifts in 3D coordinates.
Evaluating a colored asset by its geometry alone creates a massive predictive gap. In developing the Waterloo Point Cloud database, Honglei Su et al. observed in their work that viewers prioritize color and texture fidelity over geometric precision when assessing colored point clouds. Because geometry and color features play deeply complementary roles in human perception, Gabriel Meynet et al. argue in their introduction of the PCQM metric that accurate quality prediction is impossible without integrating both modalities.
The Texture-Geometry Masking Effect
The relationship between shape and appearance is not merely additive; it is interactive and non-linear. The human visual system exhibits a powerful "masking effect" where high-frequency visual information in one domain can camouflage distortions in another.
[High-Frequency, Complex Texture] ---> Masks ---> [Geometric Irregularities]
[Smooth, Untextured Surface] ---> Amplifies -> [Structural Flaws / Cracks]
When a 3D mesh is mapped with a highly detailed, complex texture (such as gravel, grass, or intricate patterns), the visual noise of the texture overpowers the eye's ability to notice underlying geometric simplifications or coordinate shifts. Conversely, on a perfectly smooth, untextured surface, even a microscopic structural flaw stands out immediately.
Bingyang Cui et al. leveraged this insight when building the SJTU-TMQA database, noting that high-quality textures effectively mask geometric artifacts from human observers. Treating these components as independent variables severely limits an objective metric. To address this, Sheyang Tang developed HybridMQA to explicitly model this cross-modal relationship, demonstrating that accounting for how smooth surfaces amplify flaws and complex textures hide them yields a much tighter alignment with human visual judgment.
Where the Human Eye Looks
This interaction fundamentally alters human attention and saliency in 3D environments. Traditional 3D saliency models focused almost exclusively on geometric features like curvature, corners, and edges to predict where a viewer would look. However, surface textures completely redraw the map of human attention.
As Kaiwei Zhang et al. show in their work on textured mesh saliency, human observers are naturally drawn to high-frequency visual details, vibrant colors, and high-contrast patterns contained entirely within the 2D texture map, rather than just the geometric "bones" of the model. When artifacts do emerge, our sensitivity to contrast dictates their visibility. For instance, the tiny fractures or "cracks" analyzed by Armin Shafiee Sarvestani et al. in their perceptual detection framework demonstrate that surface artifact visibility is heavily dependent on the underlying 3D structural context, requiring a unified approach to catch what the human eye actually registers.
The Asymmetry of Structural Integrity
While texture can mask minor geometric flaws, geometry remains the absolute foundation of a 3D asset. This creates an inherent asymmetry in how compression and optimization algorithms must treat the two modalities.
If an algorithm aggressively compresses texture data, the asset suffers from localized blurring or color degradation—defects that are visually unappealing but leave the object intact. However, if the algorithm aggressively compresses geometric data, it introduces catastrophic structural artifacts like holes, missing surfaces, or warped boundaries.
This perceptual asymmetry is quantified by Fangyu Shen et al. in their rate control algorithm for point clouds, where they reveal that overall visual reconstruction quality is approximately five times more sensitive to geometric accuracy than to color accuracy. Because structural holes are far more disruptive to a viewer than color deviations, optimization frameworks—such as the one detailed in Rate-Distortion Modeling for Bit Rate Constrained Point Cloud Compression—prioritize bit allocation heavily toward maintaining geometric structure, maximizing the perceptual return on investment.
Bridging the Dimensions
To operationalize these insights, modern objective metrics bridge the gap between 3D spatial data and 2D visual perception through multimodal fusion architectures.
3D Structural Data (Point Clouds / Meshes) ---\
---> [Cross-Modal Fusion] ---> Perceptual Score
2D Visual Projections (Rendered Viewpoints) ---/
Pure 3D representations capture spatial topology perfectly but struggle to isolate subtle texture or color artifacts. Conversely, 2D image projections capture fine-grained texture fidelity but suffer from viewpoint dependency. Modern frameworks combine the strengths of both:
- Cross-Modal Attention: Wei Sun designed MM-PCQA, a no-reference framework that uses a symmetric cross-modal attention mechanism to fuse 3D geometric features with 2D projected images, allowing the model to evaluate texture artifacts while overcoming viewpoint limitations.
- Geodesic Mapping: Instead of relying on arbitrary 2D projections, Qi Yang 0003 et al. introduced GeodesicPSIM, which measures similarity across localized patches directly along the 3D surface manifold, jointly calculating geometric deformation and texture distortion where they intersect.
By moving away from isolated feature calculations and treating geometry and appearance as an inseparable pair—a philosophy championed by Tianyu Luan et al. in their Textured Geometry Evaluation framework—the field has moved closer to replicating the holistic way humans see and evaluate the virtual world.
Key Foundational Work
Subjective Quality Database and Objective Study of Compressed Point Clouds With 6DoF Head-Mounted Display (2021) — contributed 2 concept elements to this direction.
PCQM: A Full-Reference Quality Metric for Colored 3D Point Clouds (2020) — contributed 2 concept elements to this direction.
Perceptual Quality Assessment of Colored 3D Point Clouds (2021) — contributed 2 concept elements to this direction.