The ANTsX ecosystem for quantitative biological and medical imaging

Nicholas J. Tustison, Philip A. Cook, Andrew J. Holbrook, Hans J. Johnson, John Muschelli, Gabriel A. Devenyi, Jeffrey Duda, Sandhitsu R. Das, Nicholas Cullen, Daniel L. Gillen, Michael A. Yassa, James R. Stone, James C. Gee, Brian Avants

TL;DR

This paper introduces ANTsX, an open-source ecosystem of software libraries for biological and medical image analysis. It highlights the expansion of the ANTs library beyond its origins in image registration to incorporate deep learning capabilities, particularly for cortical thickness estimation, demonstrating significant improvements in computational efficiency and comparable-to-superior accuracy.

What Problem Does This Paper Address?

Medical image analysis, especially in fields like neuroscience, relies on sophisticated tools for tasks such as image registration, segmentation, and feature extraction (e.g., cortical thickness). While established toolkits like ANTs have been instrumental, the rapid advancements in deep learning present an opportunity to enhance both the speed and accuracy of these analyses. However, integrating deep learning into existing workflows can be challenging due to a lack of standardization, disorganized code, and complex external data interfaces. This paper addresses the need for a unified, accessible, and high-performing framework that leverages deep learning to extend the capabilities of established medical imaging software.

Methods

The core of the ANTsX ecosystem is built upon the ANTs library, which has been extended with deep learning capabilities through the ANTsR (R) and ANTsPy (Python) interfaces. These interfaces connect to ANTsRNet and ANTsPyNet, which utilize TensorFlow/Keras to implement and deploy popular deep learning architectures.

Key methodological advancements include:

  • Deep Learning Analogs for Existing Pipelines: The paper focuses on developing deep learning versions of established ANTs pipelines, particularly for cortical thickness estimation. This involves replacing computationally intensive steps like deformable image registration with neural networks.
  • ANTsXNet Framework: This framework standardizes popular network architectures and trained models for various applications, ensuring cross-compatibility between R and Python.
  • Efficient Cortical Thickness Estimation: The ANTsXNet cortical thickness pipeline replaces traditional registration-based methods with deep learning models (e.g., 3D U-nets for brain extraction and segmentation). This significantly reduces processing time.
  • Longitudinal Analysis: The ecosystem supports longitudinal studies by incorporating single-subject template (SST) generation and processing, further enhanced by deep learning.
  • Bayesian Linear Mixed-Effects (LME) Modeling: Used for longitudinal performance evaluation to quantify between-subject and residual variabilities, providing an estimate of biomarker effectiveness.
  • Tukey Post-Hoc Analyses: Employed for supervised evaluation to assess diagnostic differentiation between clinical groups.

Datasets

The paper utilizes several publicly available datasets for training and evaluation:

  • Training Data: IXI, MMRR, NKI, and OASIS datasets were used for training the deep learning models for brain extraction, segmentation, and DKT parcellation. This collectively comprised over 1200 subjects with ages ranging from 4 to 94.
  • Evaluation Data:
    • Cross-sectional: IXI, MMRR, NKI, OASIS, and SRPB datasets were used for cross-sectional performance evaluation, primarily through age prediction models.
  • Longitudinal: The Alzheimer's Disease Neuroimaging Initiative (ADNI-1) dataset was used for longitudinal performance evaluation, including cognitive normals, mild cognitive impairment (LMCI), and Alzheimer's disease (AD) subjects.

Key Insights and Ideas

  • Deep Learning as a Performance Enhancer: The study demonstrates that deep learning can significantly accelerate image processing tasks (like cortical thickness estimation) without sacrificing, and often improving, accuracy compared to traditional methods.
  • Unified Ecosystem for Reproducibility: ANTsX provides a comprehensive, open-source platform that integrates preprocessing, deep learning model training, and analysis, promoting reproducibility and accessibility for researchers.
  • Bridging the Gap: ANTsX effectively bridges the gap between established, robust image analysis tools (like ANTs) and the rapidly evolving field of deep learning, making advanced DL techniques more accessible to the medical imaging community.
  • Importance of Standardization: The ANTsXNet approach emphasizes standardization of network architectures and training procedures, facilitating cross-compatibility and easier adoption of deep learning models.
  • Beyond Registration: The ANTsX ecosystem's evolution beyond its core registration capabilities highlights a trend in scientific software development towards integrated, multi-functional platforms.

Key Results

  • Computational Efficiency: The ANTsXNet cortical thickness pipeline significantly reduces processing time compared to traditional ANTs workflows. For example, the deep learning pipeline is dominated by DiReCT, which is much faster than the previous registration-heavy pipelines.
  • Age Prediction Accuracy: In cross-sectional evaluations, the ANTsXNet pipeline outperformed the classical ANTs pipeline in age prediction accuracy across multiple datasets, with mean RMSE values generally lower for ANTsXNet.
  • Longitudinal Performance: For longitudinal data, the ANTsXNetLong pipeline showed superior performance in terms of variance ratio, comparable to the specialized ANTsSST pipeline.
  • Diagnostic Differentiation: The deep learning pipelines demonstrated excellent differentiation between clinical groups (LMCI-CN and AD-CN) in longitudinal ADNI data.
  • Repeatability: Both ANTs and ANTsXNet pipelines showed high repeatability (ICC values of 0.99) for regional cortical thickness measurements.

Limitations and Future Work

  • Training Data Dependency: Deep learning models are inherently dependent on their training data. The ANTsXNet pipeline's brain extraction, for instance, may perform poorly on data with different anonymization protocols (e.g., defacing). The authors are actively working on retraining with more diverse data to mitigate this.
  • Resolution Limitations: The current ANTsXNet pipeline is limited to high-resolution acquisitions (around 1 mm isotropic) due to memory requirements of deep learning models. Future work may explore variations that can leverage lower-resolution data or alternative approaches.
  • Potential for Pipeline Variation: The authors suggest exploring a variation where Deep Atropos is used for priors, followed by traditional Atropos segmentation on high-resolution data, which could be beneficial for certain applications.
  • Longitudinal Extensions: Future work could incorporate subject ID and time information as additional inputs to the network for longitudinal studies, potentially improving performance.
  • Generalizability to Age Ranges: While performing well across the lifespan, the pipeline's performance might be hampered for specific age ranges (e.g., neonates), suggesting that traditional ANTs pipelines might offer more flexibility in such cases.

Summary updated

Abstract

The Advanced Normalizations Tools ecosystem, known as ANTsX, consists of multiple open-source software libraries which house top-performing algorithms used worldwide by scientific and research communities for processing and analyzing biological and medical imaging data. The base software library, ANTs, is built upon, and contributes to, the NIH-sponsored Insight Toolkit. Founded in 2008 with the highly regarded Symmetric Normalization image registration framework, the ANTs library has since grown to include additional functionality. Recent enhancements include statistical, visualization, and deep learning capabilities through interfacing with both the R statistical project (ANTsR) and Python (ANTsPy). Additionally, the corresponding deep learning extensions ANTsRNet and ANTsPyNet (built on the popular TensorFlow/Keras libraries) contain several popular network architectures and trained models for specific applications. One such comprehensive application is a deep learning analog for generating cortical thickness data from structural T1-weighted brain MRI, both cross-sectionally and longitudinally. These pipelines significantly improve computational efficiency and provide comparable-to-superior accuracy over multiple criteria relative to the existing ANTs workflows and simultaneously illustrate the importance of the comprehensive ANTsX approach as a framework for medical image analysis.
BibTeX
@inproceedings{tustison2021antsx,
  author = {Nicholas J. Tustison and Philip A. Cook and Andrew J. Holbrook and Hans J. Johnson and John Muschelli and Gabriel A. Devenyi and Jeffrey Duda and Sandhitsu R. Das and Nicholas Cullen and Daniel L. Gillen and Michael A. Yassa and James R. Stone and James C. Gee and Brian Avants},
  title = {{The ANTsX ecosystem for quantitative biological and medical imaging}},
  booktitle = {Scientific},
  year = {2021},
  doi = {10.1038/s41598-021-87564-6},
}
Figure