T-measure: A Topology-Consistent Metric for Binary Segmentation

pengfei zhang, Jian JiICML 2026 · OpenReview

The Flaw in Traditional Segmentation Metrics

In computer vision, evaluating the quality of a binary segmentation model typically relies on pixel-wise metrics like the Intersection over Union (IoU) or the Dice coefficient. While mathematically straightforward, these metrics suffer from "structural blindness." Because they treat every pixel as an independent entity, a model could output a shattered, fragmented mess of pixels that technically covers the correct area, and IoU would still award it a high score.

Existing "structure-aware" metrics also have significant blind spots:

  • S-measure ($S_m$): Widely used in salient object detection, it struggles with small objects and is surprisingly insensitive to "ghosting" (false positives in the background).
  • Boundary metrics (e.g., 95HD, ASD): Common in medical imaging, these are extremely sensitive to outliers and can produce highly unstable scores.

To address this, researchers have introduced the T-measure ($T_m$), a novel evaluation metric that prioritizes topological consistency. It ensures that predicted objects are not just pixel-accurate, but structurally coherent—meaning a "cat" prediction actually looks like a single, connected cat rather than a cloud of cat-colored noise.

How T-measure Works

The T-measure combines three synergistic components into a single, robust score:

  1. Pixel-level Similarity ($S_{pix}$): This serves as the foundational score. It uses a Fuzzy Jaccard Index to provide a robust baseline of global overlap between the prediction and the ground truth.
  2. Topological Integrity ($I_{topo}$): The core innovation of the metric. It uses connected component analysis to check if the prediction fragments the object. If a single object in the ground truth is represented by multiple disconnected "islands" in the prediction, $I_{topo}$ applies a heavy penalty.
  3. Boundary Alignment ($A_{bdy}$): This evaluates how well the edges of the prediction match the ground truth using a bidirectional gradient matching strategy. It includes an adaptive "tolerance band" (1–5 pixels) to account for the slight ambiguities inherent in human annotation.

The final formula integrates these components: $$T_m = S_{pix} \times [\alpha I_{topo} + (1 - \alpha) A_{bdy}]$$

Rather than guessing the weights, the authors conducted a psychophysical study with 50 participants to find the "Human Perception Range." Based on this, they set $\alpha = 0.8$, placing a higher premium on topological connectivity than on perfect edge-matching, which aligns perfectly with human visual preferences.

Key Insights and Performance

To ensure cross-task generalization, the researchers validated $T_m$ across four diverse domains: salient object detection (DUTS), camouflaged object detection (COD10K), industrial defect detection (CrackSeg9k), and medical image segmentation (ISIC2017). They also introduced a Meta-Measure validation framework to test how metrics respond to catastrophic model failures.

1. The Fragmentation Test

In a standout experiment, the authors took a high-quality prediction and "chopped" it with 40 random lines. Traditional metrics like $F_m$ only dropped by 3.1%, falsely suggesting the shattered mask was still highly accurate. In contrast, $T_m$ plummeted by 84.3%, correctly identifying the prediction as semantically useless.

2. Fixing Small Object Bias

$T_m$ resolves a major flaw in the S-measure. If a model predicts an empty black mask for a small object, S-measure might still award a score of ~0.48 because it correctly "classified" the vast background. $T_m$ correctly drops to 0.0.

3. Human Alignment and Efficiency

  • Human Preference: When $T_m$ and the popular S-measure disagreed on which of two predictions was better, human judges agreed with $T_m$ 70.7% of the time.
  • Perceptual Consistency: $T_m$ achieved top-tier correlation with LPIPS (a deep-learning-based perceptual similarity metric).
  • Speed: Despite its structural complexity, $T_m$ runs at 24.6 FPS on a standard CPU. While slower than basic IoU, it is significantly faster than advanced metrics like $F_\beta^w$ (11 FPS) or Hausdorff Distance (13.5 FPS), making it highly practical for real-world validation pipelines.

Limitations and Future Potential

Because $T_m$ is modular, it doubles as a diagnostic tool. For instance, a high $A_{bdy}$ but low $I_{topo}$ tells a developer their model captures local edges well but fails at global connectivity.

Currently, the main limitation of $T_m$ is a binarization requirement. While $S_{pix}$ and $A_{bdy}$ work on raw probability maps, the $I_{topo}$ component requires binarizing the prediction (e.g., using a 0.5 threshold) to perform connected component analysis.

Looking forward, the authors suggest that $T_m$ could be adapted into a loss function. If models can be trained to optimize for $T_m$ directly during the learning phase, we may soon see a new generation of segmentation models that are inherently resistant to structural fragmentation.

Summary updated

Abstract

Evaluation metrics establish a standard assessment framework for models, playing a pivotal role in model optimization and advancement. However, widely adopted pixel-wise metrics (e.g., IoU, Dice) rely heavily on pixel-level statistics, often failing to capture the structural integrity of predictions. While the S-measure (S-m) incorporates structural perception to some extent, it struggles to differentiate critical structural violations and remains insensitive to background false positives and small objects. To address these limitations, we propose the Topology-aware measure (T-m), a novel metric designed to explicitly quantify topological consistency. T-m employs the Fuzzy Jaccard Index as a foundational score, integrates a Topological Integrity term (I-topo) to penalize critical structural fragmentation, and utilizes a Boundary Alignment term (A-bdy) to evaluate boundary alignment. These three components synergize to achieve robust evaluation of prediction maps at the topological level. We establish a rigorous Meta-Measure validation framework and benchmark our method against nine mainstream metrics across diverse complex scenarios. Extensive experiments demonstrate that T-m performs exceptionally in downstream tasks and maintains high consistency with human visual perception.
BibTeX
@inproceedings{zhang2026t,
  author = {pengfei zhang and Jian Ji},
  title = {{T-measure: A Topology-Consistent Metric for Binary Segmentation}},
  booktitle = {International Conference on Machine Learning},
  year = {2026},
  url = {https://openreview.net/forum?id=tRpJyIzxDj},
}
Figure