3D Visualization, Skeletonization and Branching Analysis of Blood Vessels in Angiogenesis

Vignesh Ramakrishnan, Rebecca Schönmehl, Annalena Artinger, Lina Winter, Hendrik Böck, Stephan Schreml, Florian Gürtler, Jimmy Daza, Volker H. Schmitt, Andreas Mamilos, Pablo Arbeláez, Andreas Teufel, Tanja Niedermair, Ondřej Topolčan, Marie Karlíková, Samuel Sossalla, Christoph B. Wiedenroth, Markus Rupp, Christoph BrochhausenInternational Journal of Molecular Sciences 2023 · DOI

TL;DR

This paper presents a novel computational pipeline that reconstructs and analyzes 3D blood vessel networks from standard histology slides. By combining deep learning for segmentation and advanced image registration, the method creates detailed 3D models, enabling new insights into angiogenesis and vascular diseases.

What Problem Does This Paper Address?

Understanding angiogenesis, the formation of new blood vessels, is crucial for many biological processes, from wound healing to tumor growth and metastasis. While we can observe blood vessels in 2D tissue sections, visualizing their complex 3D structure and analyzing their branching patterns has been a significant challenge. This lack of detailed 3D information limits our ability to fully grasp the intricate mechanisms of vascular development and its role in various diseases. Researchers need a robust and automated way to reconstruct these 3D vascular networks from readily available histological data to advance our understanding and diagnostic capabilities.

Methods

The authors developed a multi-stage pipeline to achieve 3D blood vessel reconstruction and analysis:

  • Image Segmentation: A U-net neural network was employed to accurately segment blood vessels from tissue images. This deep learning approach is known for its effectiveness in medical image segmentation tasks.
  • Image Registration: This is a critical step to align sequential tissue slices. The pipeline uses a two-stage approach:
  • Coarse Registration: An image-intensity optimization technique (Thevenaz's algorithm) is used for initial alignment.
  • Fine-tuning Registration: A neural network incorporating Spatial Transformers is then used to refine the alignment, achieving highly accurate registration.
  • 3D Model Reconstruction: The aligned segmented masks (representing blood vessels) are interpolated using bilinear interpolation to create a continuous 3D model.
  • Skeletonization and Analysis: A skeletonization algorithm (Lee's algorithm) is applied to the 3D model. This process reduces the vessel structures to a one-pixel-wide representation, allowing for detailed analysis of branching patterns, main branch lengths, and the number of sub-branches.

Datasets

The study utilized two sets of human granulation tissue samples:

  • H&E-stained tissue: A set of eight consecutively sliced tissue sections.
  • CD31-stained tissue: A set of ten consecutively sliced tissue sections.

The authors manually annotated regions of interest (ROIs) on these slides to serve as Ground-Truth (GT) data for training and validation. The GT data was used to generate GT masks for the segmentation algorithm.

Key Insights and Ideas

  • Deep Learning for Segmentation is Key: The U-net architecture demonstrated significantly superior performance in blood vessel segmentation compared to traditional methods like Otsu's, highlighting the power of deep learning in this domain.
  • Coarse-to-Fine Registration is Crucial: Achieving precise alignment of sequential tissue slices is paramount for accurate 3D reconstruction. The combination of a classical coarse registration method with a deep learning-based fine-tuning step proved highly effective.
  • Skeletonization Unlocks Branching Insights: Reducing the 3D vascular models to skeletons is an elegant way to extract quantitative data about vessel architecture, which is difficult to obtain from raw 3D models.
  • Staining Method Independence: The pipeline's success with both H&E and CD31 staining suggests its adaptability to different tissue preparation methods, broadening its potential applications.

Key Results

  • Segmentation Accuracy: The U-net model achieved high dice score accuracies of 92.1% for H&E-stained tissues and 91.7% for CD31-stained tissues, significantly outperforming the traditional Otsu method (67.5% and 56.1%, respectively).
  • Registration Improvement: The coarse-to-fine registration pipeline substantially improved image alignment. For instance, the average SSIM for the H&E sample increased from 29.2% to 57.0% after fine-tuning, and the mutual dice score for consecutive image pairs jumped from around 3.8% to over 32%. Similar improvements were observed for CD31 samples.
  • 3D Model Generation: The pipeline successfully generated 3D models of blood vessels from both H&E and CD31 stained tissues, allowing for visualization and subsequent skeletonization.
  • Branching Analysis: The skeletonized models enabled the quantification of branching statistics, including main branch lengths, which were successfully visualized and analyzed.

Limitations and Future Work

  • Tissue Destructive Methods: The current approach relies on histological methods that are inherently destructive to the tissue.
  • Staining Quality: The accuracy of the 3D reconstruction is dependent on the careful and reliable staining of the tissue sections.
  • Potential for Local Minima: Coarse registration algorithms can sometimes converge to local minima, potentially affecting alignment.
  • Future Directions:
    • Investigating Generative Adversarial Networks (GANs) for even more robust image registration.
  • Exploring advanced deep learning-based interpolation methods to further enhance visualization quality.
  • Applying the pipeline to radiomics data for non-invasive analysis.
  • Utilizing the pipeline for 3D reconstruction of blood vessels in various disease contexts, such as endomyocardial tissue or tumor vasculature, and for analyzing wound healing.
  • Leveraging biobanked heart tissues for 3D reconstruction.

Summary updated

Abstract

Angiogenesis is the process of new blood vessels growing from existing vasculature. Visualizing them as a three-dimensional (3D) model is a challenging, yet relevant, task as it would be of great help to researchers, pathologists, and medical doctors. A branching analysis on the 3D model would further facilitate research and diagnostic purposes. In this paper, a pipeline of vision algorithms is elaborated to visualize and analyze blood vessels in 3D from formalin-fixed paraffin-embedded (FFPE) granulation tissue sections with two different staining methods. First, a U-net neural network is used to segment blood vessels from the tissues. Second, image registration is used to align the consecutive images. Coarse registration using an image-intensity optimization technique, followed by finetuning using a neural network based on Spatial Transformers, results in an excellent alignment of images. Lastly, the corresponding segmented masks depicting the blood vessels are aligned and interpolated using the results of the image registration, resulting in a visualized 3D model. Additionally, a skeletonization algorithm is used to analyze the branching characteristics of the 3D vascular model. In summary, computer vision and deep learning is used to reconstruct, visualize and analyze a 3D vascular model from a set of parallel tissue samples. Our technique opens innovative perspectives in the pathophysiological understanding of vascular morphogenesis under different pathophysiological conditions and its potential diagnostic role.
BibTeX
@article{ramakrishnan2023d,
  author = {Vignesh Ramakrishnan and Rebecca Schönmehl and Annalena Artinger and Lina Winter and Hendrik Böck and Stephan Schreml and Florian Gürtler and Jimmy Daza and Volker H. Schmitt and Andreas Mamilos and Pablo Arbeláez and Andreas Teufel and Tanja Niedermair and Ondřej Topolčan and Marie Karlíková and Samuel Sossalla and Christoph B. Wiedenroth and Markus Rupp and Christoph Brochhausen},
  title = {{3D Visualization, Skeletonization and Branching Analysis of Blood Vessels in Angiogenesis}},
  journal = {International Journal of Molecular Sciences},
  year = {2023},
  doi = {10.3390/ijms24097714},
}
Figure