Embracing the Disharmony in Heterogeneous Medical Data

Rongguang Wang, Pratik Chaudhari, Christos Davatzikos

TL;DR

This paper introduces a novel framework for "embracing" domain shift in medical imaging, rather than trying to eliminate it. By pre-training a model on source data and then adapting it to target data using auxiliary tasks (like predicting age or gender), they achieve significant improvements in disease classification accuracy, even when target labels are unavailable.

What Problem Does This Paper Address?

Medical imaging data is notoriously heterogeneous. Differences in scanners, patient populations, and acquisition protocols across different hospitals or research sites create a "domain shift." This shift severely degrades the performance of machine learning models trained on data from one source when applied to data from another. Traditional methods aim to harmonize data by learning domain-invariant representations, but this often comes at the cost of reduced predictive accuracy. The authors tackle this challenge by proposing a framework that leverages this disharmony to improve model performance.

Methods

The core of the proposed method involves a two-pronged approach:

  • Intra-Study Domain Generalization: For situations where data within a single study comes from different subgroups (e.g., age ranges, scanner types), they fine-tune a pre-trained model. Instead of simply retraining on a subgroup, they use a "proximal" term in the loss function that encourages the adapted model's weights to stay close to the original pre-trained weights. This helps prevent overfitting to the smaller subgroup data while still adapting to its specific characteristics.
  • Inter-Study Domain Generalization (with Auxiliary Tasks): When dealing with entirely different studies (e.g., data from different hospitals), and crucially, when ground-truth labels for the target study are unavailable, they employ "auxiliary tasks." The model is trained to perform both the primary task (e.g., disease classification) and auxiliary tasks (e.g., predicting age, gender, or race) simultaneously on the source data. Then, the feature extractor is adapted to the target domain using these auxiliary tasks. This allows the model to learn features that are useful for the target domain's characteristics, even without direct labels for the primary task.

Datasets

The authors validate their framework on two large-scale 3D T1-weighted brain MRI datasets:

  • iSTAGING Consortium: This dataset includes data from multiple sites (ADNI, PENN, AIBL) for classifying Alzheimer's disease (AD) and cognitive normal (CN) individuals.
  • PHENOM Consortium: This dataset includes data from Penn, China, and Munich for classifying Schizophrenia (SCZ) and normal control (NC) individuals.

The datasets exhibit significant heterogeneity in terms of scanner manufacturers, magnetic strength, and acquisition protocols.

Key Insights and Ideas

  • Embracing Disharmony: The central idea is that instead of fighting domain shift, we can leverage it. By adapting models to specific subgroups or using related auxiliary tasks, we can achieve better performance than trying to create a single, invariant representation.
  • Auxiliary Tasks as Bridges: Auxiliary tasks, especially those using easily obtainable demographic information (age, gender, race), act as powerful bridges to transfer knowledge between domains, even without target labels. This is crucial for real-world medical imaging where labeling is expensive.
  • Proximal Regularization for Intra-Study Adaptation: The "proximal" term in the loss function is key for adapting to subgroups within a study. It prevents the model from deviating too far from the well-generalized source model, leading to more stable and accurate predictions for specific subgroups.
  • Age Prediction is a Stronger Auxiliary Task: The paper consistently shows that age prediction as an auxiliary task yields better results than sex classification. This is attributed to age being a more complex and informative covariate that is closely linked to brain aging and neurodegeneration.

Key Results

  • Substantial Accuracy Improvements: The proposed framework consistently achieved higher accuracies and reduced variance compared to baseline models (e.g., models trained only on source data and tested on target data) in both intra-study and inter-study scenarios.
  • Effective Domain Generalization: For inter-study generalization, the adapted models showed significant improvements, bridging the large accuracy gaps observed with baseline models. For instance, in AD classification, accuracy improved from 78.84% to 92.14% when transferring from ADNI-1 to PENN.
  • Reduced Health Care Disparities: The authors highlight how their transfer learning approach can help mitigate health care disparities by improving model performance on underrepresented groups (e.g., older age groups, specific ethnic groups).
  • Robustness to Scanner and Population Variations: The method demonstrated effectiveness in adapting to variations in scanner types, age groups, and race groups.

Limitations and Future Work

  • Hyper-parameter Sensitivity: While the paper provides a hyper-parameter selection study, the performance can still be sensitive to the choice of parameters like $\alpha$ and $\beta_a$.
  • Future Work: The authors suggest exploring the generalizability of self-supervised learning, such as contrastive learning, for situations where labels are unavailable, as a promising direction for future research. They also note that the proposed framework can be adapted to other auxiliary tasks relevant to different applications.

Summary updated

Abstract

Domain shift, the mismatch between training and testing data characteristics, causes significant degradation in the predictive performance in multi-source imaging scenarios. In medical imaging, the heterogeneity of population, scanners and acquisition protocols at different sites presents a significant domain shift challenge and has limited the widespread clinical adoption of machine learning models. Harmonization methods which aim to learn a representation of data invariant to these differences are the prevalent tools to address domain shift, but they typically result in degradation of predictive accuracy. This paper takes a different perspective of the problem: we embrace this disharmony in data and design a simple but effective framework for tackling domain shift. The key idea, based on our theoretical arguments, is to build a pretrained classifier on the source data and adapt this model to new data. The classifier can be fine-tuned for intra-site domain adaptation. We can also tackle situations where we do not have access to ground-truth labels on target data; we show how one can use auxiliary tasks for adaptation; these tasks employ covariates such as age, gender and race which are easy to obtain but nevertheless correlated to the main task. We demonstrate substantial improvements in both intra-site domain adaptation and inter-site domain generalization on large-scale real-world 3D brain MRI datasets for classifying Alzheimer's disease and schizophrenia.
BibTeX
@article{wang2022embracing,
  author = {Rongguang Wang and Pratik Chaudhari and Christos Davatzikos},
  title = {{Embracing the Disharmony in Heterogeneous Medical Data}},
  journal = {Medical Image Analysis},
  year = {2022},
  url = {https://openreview.net/forum?id=ymfd7Lk8Fzs},
}
Figure