A topological encoding convolutional neural network for segmentation of 3D multiphoton images of brain vasculature using persistent homology

Mohammad Haft-Javaherian, Martin Villiger, Chris B. Schaffer, Nozomi Nishimura, Polina Golland, Brett E. BoumaWorkshops @ CVPR 2020 · OpenReview

Abstract

Understanding the complex 3D network of brain blood vessels is vital for researching cognitive disorders linked to vascular dysfunction. However, manual segmentation of high-resolution 3D multiphoton microscopy (MPM) images is a significant bottleneck. While convolutional neural networks (CNNs) have automated this process, they often fail to maintain the structural connectivity of vessels, especially in noisy, deep-tissue images. We present a novel topological encoding CNN that utilizes persistent homology to ensure structural integrity. Our model outperforms existing state-of-the-art methods in Dice coefficient and produces segmentations that better mimic the topological characteristics of manual ground truth.

Introduction

Mapping the brain's dense vasculature is essential for studying neurodegenerative diseases like Alzheimer’s. 3D multiphoton microscopy provides the necessary sub-micron resolution, but the resulting data is massive and difficult to process. Standard CNNs, while efficient, often produce "broken" vessels or artificial holes in regions with low signal-to-noise ratios. These topological errors occur because traditional loss functions, like Dice loss or cross-entropy, focus on local pixel-level accuracy rather than global structural connectivity.

Methodology

The proposed approach addresses these limitations by incorporating Persistent Homology (PH)—a tool from algebraic topology—directly into the training process.

Architecture and Loss Function

The model builds upon the DeepVess architecture, utilizing 3D convolutional layers to capture spatial context. The key innovation is a hybrid loss function that balances three distinct objectives:

  1. Cross-Entropy: For precise voxel-wise classification.
  2. Generalized Dice Loss: To maximize the overall overlap between predicted and ground-truth volumes.
  3. Topological Loss: A differentiable term based on the Wasserstein-1 distance between persistence diagrams. This term penalizes the creation of disconnected components or artificial loops.
The proposed CNN architecture integrates a topological loss term alongside standard voxel-wise and overlap-based losses.

Computational Efficiency

Calculating PH is computationally intensive. To make training feasible, the authors implemented a sampling strategy where the topological loss is only calculated on 10% of the samples in each mini-batch. Furthermore, by optimizing the field of view to a $21 \times 21 \mu m^2$ region, the target topology could be simplified to a single connected component with no holes, streamlining the mathematical constraints.

Key Results

The model was evaluated using the DeepVess dataset, which contains manually labeled 3D MPM images from 24 mice. The results demonstrate significant improvements in both quantitative metrics and qualitative structural integrity.

Quantitative Performance

The topological encoding model achieved a Dice coefficient of 86.5%, significantly outperforming the original DeepVess (81.6%) and 3D U-Net models. It also showed greater robustness to imaging depth; while standard models' performance degrades as signal quality drops deeper in the brain, the proposed model maintained consistent accuracy.

Comparison of Dice coefficients across different models, showing that the proposed topological encoding model ('Ours') achieves superior performance.

Qualitative Improvements

Beyond numerical scores, the model produced segmentations with far fewer discontinuities. By penalizing "broken" vessels, the topological loss ensures that the resulting 3D maps are suitable for downstream applications, such as blood flow simulations, which require continuous vessel paths.

Conclusion

By bridging the gap between deep learning and algebraic topology, this research provides a robust framework for segmenting complex biological structures. The use of persistent homology as a regularizer ensures that the network "understands" the fundamental connectivity of the objects it segments. Future work will explore generalizing this approach to other tubular structures, such as neuronal networks and pulmonary airways.

Source: A topological encoding convolutional neural network for segmentation of 3D multiphoton images of brain vasculature using persistent homology (CVPR 2020 Workshop). Code available on GitHub.

Summary updated

Abstract

The clinical evidence suggests that cognitive disorders are associated with vasculature dysfunction and decreased blood flow in the brain. Hence, a functional understanding of the linkage between brain functionality and the vascular network is essential. However, methods to systematically and quantitatively describe and compare structures as complex as brain blood vessels are lacking. 3D imaging modalities such as multiphoton microscopy enables researchers to capture the network of brain vasculature with high spatial resolutions. Nonetheless, image processing and inference are some of the bottlenecks for biomedical research involving imaging, and any advancement in this area impacts many research groups. Here, we propose a topological encoding convolutional neural network based on persistent homology to segment 3D multiphoton images of brain vasculature. We demonstrate that our model outperforms state-of-the-art models in terms of the Dice coefficient and it is comparable in terms of other metrics such as sensitivity. Additionally, the topological characteristics of our model's segmentation results mimic manual ground truth. Our code and model are open source at https://github.com/mhaft/DeepVess.
BibTeX
@inproceedings{haftjavaherian2020topological,
  author = {Mohammad Haft-Javaherian and Martin Villiger and Chris B. Schaffer and Nozomi Nishimura and Polina Golland and Brett E. Bouma},
  title = {{A topological encoding convolutional neural network for segmentation of 3D multiphoton images of brain vasculature using persistent homology}},
  booktitle = {Workshops @ CVPR},
  year = {2020},
  url = {https://openreview.net/forum?id=mjgg0s0LUMAv},
}
Figure