Perceptual Crack Detection for Rendered 3D Textured Meshes

Armin Shafiee Sarvestani, Wei Zhou, Zhou Wang

Introduction

In the evolving landscape of digital media, 3D textured meshes have become the standard representation for objects in virtual reality (VR), gaming, and medical visualization. However, the process of compressing, transmitting, and rendering these meshes often introduces specific visual artifacts known as "cracks." These are tiny fractures or holes on the object's surface, typically resulting from errors in vertex positioning or UV map quantization during processing.

Traditional Image Quality Assessment (IQA) metrics, such as the Structural Similarity Index (SSIM) or Peak Signal-to-Noise Ratio (PSNR), are designed for general natural images. They often struggle to differentiate these sharp, localized 3D cracks from intended textures or general noise. This paper introduces Perceptual Crack Detection (PCD), a specialized method designed to identify and localize these artifacts by mimicking the Human Visual System's (HVS) sensitivity to contrast and edges.

The PCD Methodology

The PCD framework operates on 2D snapshots of 3D objects, utilizing a four-stage image-based pipeline to generate a "crack likelihood map."

1. Truncated Absolute Difference (TAD)

The process begins by calculating the pixel-wise difference between a reference frame and a distorted frame. To ensure the model focuses on significant errors rather than negligible noise, a truncation threshold (0.1) is applied, discarding minor variations that are unlikely to be perceived as cracks by a human observer.

2. Contrast Modulation (Visual Masking)

A key insight of the HVS is that artifacts are more visible in smooth, low-contrast areas than in busy, highly textured regions—a phenomenon known as visual masking. PCD normalizes the error map based on the local contrast of the reference image, effectively "masking" artifacts that would be hidden by complex textures, such as gravel or foliage.

3. Laplacian Enhancement

Cracks are fundamentally high-frequency edge artifacts. To account for the HVS’s extreme sensitivity to sharp changes, the map is multiplied by the absolute Laplacian of the distorted frame. This step highlights the sharp, unexpected edges that characterize cracks, distinguishing them from smoother geometric distortions.

4. Sigmoid Non-linearity

Finally, the values are passed through a sigmoid function to produce a crack likelihood map (ranging from 0 to 1). This reflects "psychometric saturation," where the human ability to distinguish between levels of degradation plateaus once the artifact reaches a certain severity.

Integration and Weighting

Beyond standalone detection, PCD is designed to enhance existing IQA models. The authors propose a weighting mechanism that uses the crack likelihood map to prioritize specific pixels during quality score calculations. By directing established metrics like SSIM or FSIM to "look closer" at identified crack regions, the overall accuracy of these metrics in predicting human quality ratings is significantly improved.

Experimental Evaluation

The study evaluated PCD using two major public datasets:

  • Nehme et al. Dataset: A large-scale collection featuring 55 source models and over 343,000 distorted stimuli.
  • Textured Static Mesh Dataset (TSMD): Comprising 42 source models and 210 distorted stimuli evaluated via human subjective testing.

Key Results

The integration of PCD led to substantial performance gains across standard metrics. For instance, the Spearman Rank Correlation Coefficient (SRCC) for lumaPSNR on the Nehme dataset increased from 0.469 to 0.657. Furthermore, the standalone Crack Artifact Score (CAS) derived from PCD outperformed most established IQA models in predicting human perception of 3D mesh quality.

From a computational standpoint, the algorithm is highly efficient. Running at approximately 100 fps for $650 \times 550$ resolutions and 16 fps for 1080p video on a standard consumer CPU (Intel i7-12700K), PCD is well-suited for real-time quality monitoring in production pipelines.

Discussion and Limitations

While PCD represents a significant step forward, it has certain limitations. As an image-based metric, it is viewpoint-dependent; cracks hidden from the current camera angle remain undetected unless multiple viewpoints are rendered. Additionally, it is currently a "Full-Reference" model, requiring the original undistorted mesh for comparison.

Future work may involve developing "No-Reference" versions of the tool and exploring more sophisticated ways to combine crack detection with geometry-based metrics to further refine accuracy in complex 3D environments.

Summary updated

BibTeX
@inproceedings{sarvestani2024perceptual,
  author = {Armin Shafiee Sarvestani and Wei Zhou and Zhou Wang},
  title = {{Perceptual Crack Detection for Rendered 3D Textured Meshes}},
  booktitle = {QoMEX},
  year = {2024},
  doi = {10.1109/qomex61742.2024.10598253},
}
Figure