TL;DR
This paper introduces resilience, a training-free uncertainty estimation method designed specifically for Neural Cellular Automata (NCAs) in medical image segmentation. By treating the NCA as a dynamical system and measuring how well a prediction recovers from small state perturbations, resilience reliably flags incorrect or unstable segmentations without requiring model retraining or architectural changes.
What Problem Does This Paper Address?
Medical image segmentation requires reliable uncertainty estimation to prevent silent failures in clinical settings. While standard uncertainty methods (like Monte Carlo Dropout or Deep Ensembles) are tailored for traditional feed-forward architectures, they fail to capture the unique properties of iterative models like Neural Cellular Automata (NCAs).
NCAs generate predictions dynamically by repeatedly applying a local update rule to a spatial state tensor. Consequently, a final segmentation mask might look sharp but actually be highly unstable if the underlying system hasn't converged to a stable attractor. This paper addresses the lack of NCA-native uncertainty metrics by looking directly at the stability of these inference dynamics.
Methods
The authors propose resilience, a "perturb-and-recover" approach that treats confident predictions as stable attractors in a dynamical system. The process unfolds in four simple steps:
- Prediction: Run the NCA normally for $T$ steps to get the final latent state $S_T$ and the initial segmentation mask $m$.
- Perturbation: Inject small Gaussian noise into the latent state to create a perturbed state $\tilde{S}_T$.
- Relaxation: Allow the NCA to evolve for an additional $T'$ steps from this perturbed state to get a relaxed mask $m'$.
- Quantification: Compute uncertainty as $u_{res} = 1 - \text{IoU}(m, m')$.
If the system recovers the original mask (high Intersection over Union, low uncertainty), the prediction lies within a stable attractor basin. If the mask changes substantially (high uncertainty), the prediction is deemed unstable and untrustworthy. Unlike traditional uncertainty baselines, resilience requires zero architectural modifications and zero retraining.
Datasets
The method was evaluated across five binary medical image segmentation benchmarks spanning multiple imaging modalities:
- ClinicDB: Endoscopic frames for polyp segmentation.
- DSB 2018: Data Science Bowl benchmark for nuclei segmentation.
- ISIC 2017: Skin lesion dermoscopy segmentation.
- Kvasir-SEG: Colonoscopy polyp dataset.
- NuInsSeg: Histology tissue domains for nuclei segmentation.
To assess uncertainty under distribution shifts, the authors also evaluated performance on augmented test sets for ClinicDB and Kvasir-SEG by applying geometric and photometric perturbations to the original samples.
Key Insights and Ideas
- Uncertainty as Instability: Instead of relying on pixel-wise softmax probabilities or network variance, uncertainty can be framed as a system's sensitivity to state perturbations.
- Exploiting Iterative Structure: Because resilience exploits the intrinsic iterative nature of NCAs, it provides a strong uncertainty signal completely post-hoc.
- Complementary Failure Modes: The authors note that different baselines capture different issues (e.g., stoptime captures sensitivity to rollout length, flicker captures temporal oscillations), but resilience acts as an excellent general-purpose proxy for overall segmentation trustworthiness.
Key Results
Resilience consistently outperformed or matched five strong baselines (including Test-Time Augmentation (TTA) and single-pass entropy) across selective prediction metrics ($\Delta$Dice@90, AURC) and ranking metrics (AUROC, AUPRC).
- Failure Detection: On the ISIC 2017 dataset, resilience achieved an AUROC of 0.910, significantly outperforming TTA (0.758) and single-pass entropy (0.534).
- Selective Prediction: On ClinicDB, resilience delivered a $\Delta$Dice@90 of 8.13%, meaning that keeping only the 90% most confident images yields a massive boost in segmentation quality compared to using the entire dataset.
Limitations and Future Work
- Fixed Hyperparameters: The authors used a fixed perturbation magnitude ($\sigma = 0.02$) and relaxation length ($T' = 12$) across all datasets. Performance could likely be optimized further with dataset-specific tuning.
- Hybrid Approaches: While resilience is a strong default, combining it with other baseline metrics (like flicker or stoptime) might yield even more robust, hybrid uncertainty estimators.
- Beyond Segmentation: Since resilience is readout-agnostic and only probes the latent state dynamics, a promising future direction is applying it to other NCA tasks, such as image classification and generative modeling.
Summary updated