Abstract
Deep learning models for medical image segmentation often suffer from "brittleness," failing when applied to data from different scanners or sequences. We introduce BayeSeg, a novel Bayesian framework designed to enhance model generalizability. BayeSeg decomposes images into domain-invariant "contours" (structural edges) and domain-specific "basis" components (intensity/appearance). By conditioning the segmentation label solely on the structural contours, the model learns to ignore volatile intensity statistics. Our results demonstrate a significant performance leap, with BayeSeg achieving an average Dice score of 0.701 on unseen T2 MRI sequences compared to 0.118 for a standard U-Net.
Introduction
While deep learning has transformed medical imaging, its clinical utility is often limited by poor generalizability. A model trained on a specific MRI sequence, such as Late Gadolinium Enhancement (LGE), frequently fails when deployed on T2-weighted images because the intensity distributions and contrasts differ significantly, even though the underlying anatomy is identical. Manually re-labeling data for every new sequence is impractical. The challenge lies in creating a model that captures the anatomical "essence" (shape) while remaining invariant to the imaging modality's "noise" (appearance).
Methodology
The BayeSeg framework addresses this challenge through a combination of image decomposition, Bayesian statistics, and deep learning.
Image Decomposition and PGM
The core concept is to treat an input image ($y$) as a composition of two distinct parts: a contour ($x$), representing structural edges, and a basis ($n$), representing intensity and appearance. The authors formalize this using a Probabilistic Graphical Model (PGM) where the segmentation label ($z$) depends exclusively on the contour ($x$). Since anatomy is consistent across sequences while intensity varies, this decoupling fosters robustness.
Deep Bayesian Implementation
The framework is implemented in two stages:
- Decomposition Stage: Two Residual Networks (ResNets) infer the posterior distributions of the contour and the basis components.
- Segmentation Stage: A U-Net takes a random sample from the inferred contour distribution to generate the final segmentation label.
To handle the mathematical intractability of exact Bayesian posteriors, the authors utilize Variational Bayesian (VB) inference. They minimize a variational loss that encourages the neural networks to accurately approximate the true statistical distributions of shapes and labels.
Experimental Results
The researchers evaluated BayeSeg using the MSCMRseg (multi-sequence cardiac MR) and ACDC (cine-MRI) datasets. The primary test for generalizability involved training the model on LGE sequences and testing it on unseen T2 sequences.
Performance on Unseen Data
BayeSeg demonstrated a staggering improvement in cross-sequence generalizability. While a standard U-Net achieved a meager average Dice score of 0.118 on T2 images, BayeSeg reached 0.701. This nearly 0.6 improvement highlights the effectiveness of the contour-based decomposition.
Cross-Site Stability
In cross-site evaluations (transferring from MSCMRseg to ACDC), BayeSeg exhibited the lowest performance drop, measured by the 'G' metric (G = 0.093). This indicates that the model is more robust to variations in scanner types and hospital protocols than traditional deterministic or probabilistic architectures like PU-Net.
Discussion and Conclusion
BayeSeg's success stems from the explicit modeling of labels as a function of structural contours. By forcing the segmentation network to prioritize "edges" over "pixels," the model naturally ignores the contrast shifts that typically degrade performance. Furthermore, by learning distributions rather than point estimates, the model handles ambiguity more effectively and avoids overconfident failures on out-of-distribution data.
While BayeSeg represents a significant step toward reliable, generalizable medical image segmentation, future work may address the computational overhead of the multi-network architecture and explore the framework's efficacy across different imaging modalities, such as CT to MRI transitions.
Summary updated