Automated assessment of human engineered heart tissues using deep learning and template matching for segmentation and tracking

José M. Rivera‐Arbeláez, Danjel Keekstra, Carla Cofiño‐Fabrés, Tom Boonen, Milica Dostanić, Simone A. ten Den, Kim Vermeul, Massimo Mastrangeli, Albert van den Berg, Loes I. Segerink, Marcelo C. Ribeiro, Nicola Strisciuglio, Robert PassierBioengineering 2023 · DOI

TL;DR

Researchers have developed HAARTA, a novel software that automates the analysis of engineered heart tissues (EHTs) by combining deep learning for segmentation and template matching for precise tracking. This tool promises to standardize and accelerate drug discovery and disease modeling by providing accurate, high-throughput functional assessments of EHTs.

What Problem Does This Paper Address?

The pharmaceutical industry faces significant challenges with drug development, particularly concerning cardiovascular toxicity and efficacy. Traditional animal models are expensive, time-consuming, and don't always accurately predict human responses. Human in vitro models, such as engineered heart tissues (EHTs) derived from patient-derived pluripotent stem cells, offer a more relevant alternative. However, analyzing the contractile properties of these EHTs—a key indicator of their function and response to drugs—has been hampered by a lack of robust, automated, and standardized analysis tools. Existing methods often rely on relative measurements, are susceptible to noise and artifacts, and require manual intervention, limiting their scalability for high-throughput screening and longitudinal studies.

Methods

The core of HAARTA lies in its innovative approach to segmenting and tracking EHTs from brightfield videos.

  • Segmentation: The software employs a deep learning model, specifically Deeplabv3 with a ResNet-101 backbone, to automatically identify and segment key regions within the EHT video frames. These regions include the tissue itself, the outer pillar (where the tissue is anchored), the inner pillar, and the background. This segmentation is crucial for defining the areas of interest for subsequent analysis.
  • Template Matching with Sub-Pixel Precision: To track the movement of the EHT during contraction, HAARTA utilizes template matching. The outer pillar region, identified during segmentation, is used as a template. The algorithm then searches for this template in subsequent frames. To achieve high accuracy, it employs sub-pixel interpolation techniques. This allows the detection of movements smaller than a single pixel, significantly improving the precision of the tracking.
  • Combined Approach: The synergy between accurate deep learning-based segmentation and precise sub-pixel template matching is what makes HAARTA robust. The segmentation provides a reliable template, and the sub-pixel tracking ensures that even subtle movements are captured, leading to a more accurate and standardized measurement of contractile properties.

Datasets

  • Training Data: The study used a dataset of 81 EHT brightfield videos for training the segmentation models. Specifically, ground truth segmentation maps were manually annotated for these videos.
  • Evaluation Data: The software was evaluated using a dataset of EHTs from three different human pluripotent stem cell (hPSC) lines. Additionally, simulation videos were created to rigorously test the tracking accuracy under controlled conditions with varying levels of noise.

Key Insights and Ideas

  • Deep Learning for Robust Segmentation: Using deep learning (Deeplabv3 with ResNet-101) for segmentation proved more reliable than other approaches, especially in handling variations in pillar shape and background noise, which are common challenges in brightfield microscopy of EHTs.
  • Sub-Pixel Precision is Key: The ability to track movements at a sub-pixel level is critical for accurately quantifying the subtle contractions of EHTs. This level of precision is often missed by traditional pixel-based methods.
  • Outer Pillar as a Reliable Template: The outer pillar region was identified as a more stable and reliable template for tracking compared to the inner pillar, likely due to its more consistent shape across different EHTs.
  • Standardization is Achievable: HAARTA's automated nature and consistent output format address the critical need for standardization in EHT analysis, making it easier to compare results across different experiments and labs.

Key Results

  • Accuracy: HAARTA demonstrated improved robustness and accuracy compared to the state-of-the-art method, MUSCLEMOTION. In simulations, it maintained stable tracking with up to 25% added noise, achieving an error below 0.2 pixels. With lower noise levels (up to 20%), the precision was better than 0.1 pixels.
  • Performance: The software successfully analyzed EHTs from three different hPSC lines, demonstrating its applicability across various cell sources. It also accurately captured the response of EHTs to the positive inotropic agent isoproterenol.
  • Speed: While specific computational speed comparisons are detailed in Table 2, the paper highlights the computational efficiency of the algorithm, making it suitable for analyzing large datasets.
  • Qualitative Assessment: 98% of tested videos received a score of 5 (best) in qualitative evaluation, indicating excellent tracking performance.

Limitations and Future Work

  • Segmentation Model Specificity: The authors note that the segmentation model is trained on specific EHT configurations and might require retraining for significantly different setups.
  • Noise Level Estimation: They suggest that a future improvement would be to automatically measure the video noise level before analysis to further refine the algorithm's reliability.
  • Low Displacement Scenarios: For cases with very low displacement (e.g., diseased tissues), they propose incorporating a low-pass filter on trajectories to mitigate relative error.
  • Broader Applicability: While HAARTA is designed for pillar-based contractile tissues, the authors believe the underlying principles of template matching with sub-pixel precision could be adapted for other engineered heart or muscle tissue formats.

Summary updated

Abstract

The high rate of drug withdrawal from the market due to cardiovascular toxicity or lack of efficacy, the economic burden, and extremely long time before a compound reaches the market, have increased the relevance of human in vitro models like human (patient-derived) pluripotent stem cell (hPSC)-derived engineered heart tissues (EHTs) for the evaluation of the efficacy and toxicity of compounds at the early phase in the drug development pipeline. Consequently, the EHT contractile properties are highly relevant parameters for the analysis of cardiotoxicity, disease phenotype, and longitudinal measurements of cardiac function over time. In this study, we developed and validated the software HAARTA (Highly Accurate, Automatic and Robust Tracking Algorithm), which automatically analyzes contractile properties of EHTs by segmenting and tracking brightfield videos, using deep learning and template matching with sub-pixel precision. We demonstrate the robustness, accuracy, and computational efficiency of the software by comparing it to the state-of-the-art method (MUSCLEMOTION), and by testing it with a data set of EHTs from three different hPSC lines. HAARTA will facilitate standardized analysis of contractile properties of EHTs, which will be beneficial for in vitro drug screening and longitudinal measurements of cardiac function.
BibTeX
@inproceedings{riveraarbelaez2023automated,
  author = {José M. Rivera‐Arbeláez and Danjel Keekstra and Carla Cofiño‐Fabrés and Tom Boonen and Milica Dostanić and Simone A. ten Den and Kim Vermeul and Massimo Mastrangeli and Albert van den Berg and Loes I. Segerink and Marcelo C. Ribeiro and Nicola Strisciuglio and Robert Passier},
  title = {{Automated assessment of human engineered heart tissues using deep learning and template matching for segmentation and tracking}},
  booktitle = {Bioengineering},
  year = {2023},
  doi = {10.1002/btm2.10513},
}

Loading related content...

Figure