Introduction
In medical imaging, the accurate segmentation of anatomical structures—such as brain tumors or internal organs—is vital for diagnosis and treatment planning. However, standard evaluation metrics like the Dice Score and Hausdorff Distance (HDD) often fail to capture critical surface details. While the Dice Score measures volumetric overlap, it is largely blind to surface texture. Similarly, the HDD measures the maximum distance between two surfaces but remains insensitive to the density of surface irregularities.
This paper introduces a new suite of metrics—Roughness Index (RI) and Roughness Distance (RD)—adapted from civil engineering principles to quantify surface "jaggedness." These metrics aim to improve the evaluation and post-processing of 3D medical images by detecting local surface irregularities like small spikes or holes that traditional metrics overlook.
Methodology: Adapting Surface Profile Logic
The authors look toward civil engineering, where measuring the roughness of materials like concrete is standardized using laser-scanning parameters. They adapt these concepts for the digital voxels of medical scans using the following parameters:
- Zeta ($\zeta$): Instead of a flat laser plane, the authors use the object's center of gravity as a reference point. $\zeta$ represents the distance from the center to any point on the surface.
- Roughness Index (RI): This calculates the average deviation of the surface height ($\zeta$) from the mean height within a local window. A high RI indicates a jagged, irregular surface.
- Roughness Distance (RD/ARD): While RI describes a single object, RD measures the difference in roughness between two objects, such as a ground truth mask and an AI-generated prediction.
The Rough Boolean Matrix and Surface Smoothing
Beyond evaluation, the authors propose a "Rough Boolean Matrix" approach for post-processing. By defining the segmentation space as a union of foreground and background sets, the algorithm identifies specific voxels that are "too rough" compared to their neighbors based on a threshold ($\kappa$). These voxels are then removed or filled, effectively "sanding down" the segmentation to produce a smoother, more anatomically plausible result.
Experimental Validation
The metrics were validated using controlled synthetic datasets consisting of 2D images ($100 \times 100$ pixels) and 3D volumes ($100 \times 100 \times 100$ voxels). These models featured spheres and circles modified with "spikes" and "holes" of varying frequencies and lengths. By using synthetic data with known error parameters, the authors could precisely demonstrate the limitations of traditional metrics.
Key Insights and Results
The "Hausdorff Blind Spot"
A primary finding is the visual and quantitative proof that HDD is a "coarse" metric. In experiments, a sphere with a single spike and a sphere with dozens of spikes yielded the exact same HDD score (20.0). In contrast, the proposed Average Roughness Distance (ARD) increased significantly (from 0.0592 to 0.0692), correctly reflecting the increased noise and drop in segmentation quality.
Sensitivity to Error Density
Quantitative results further highlight this sensitivity. When moving from a low-roughness 2D model to a high-roughness model, the Roughness Ratio (RR) jumped from 0.0235 to 0.1377, while the Hausdorff Distance remained unchanged at 20. This demonstrates that RI and RD can distinguish between a mostly smooth surface with a single outlier and a consistently noisy surface.
Effectiveness of Post-Processing
The proposed smoothing algorithm successfully transformed a "star-shaped" noisy segmentation back into a near-perfect circle, reducing the RI from 0.1847 to 0.0417. This suggests that the Roughness Matrix can be used not just for benchmarking, but as an active tool for improving model outputs.
Limitations and Future Work
While promising, the authors note several challenges for clinical adoption:
- Alignment Sensitivity: The calculation of $\zeta$ relies on the center of gravity. Significant misalignments between the prediction and ground truth can skew the results.
- Hyperparameter Tuning: The choice of "window size" for local roughness is critical. Optimal results were found using 3% to 10% of the image's smallest dimension, but this may vary across different anatomical structures.
- Clinical Correlation: Future research is required to determine how these roughness metrics correlate with actual clinical outcomes in complex real-world datasets, such as brain MRIs or lung CT scans.
Summary updated