TL;DR
This paper investigates the potential of Multi-Task Learning (MTL) to improve diagnostic accuracy for psychiatric conditions and genetic variants using fMRI data. While MTL showed promise for predicting age and sex across different data collection sites, it surprisingly harmed diagnostic accuracy when applied to a combined set of psychiatric conditions and CNVs, suggesting that large sample sizes are crucial for successful MTL in this domain.
What Problem Does This Paper Address?
Machine learning (ML) models show promise for automatically diagnosing psychiatric conditions from fMRI data. However, a major hurdle is the poor generalization of these models to independent datasets, often leading to a sharp drop in performance. This is partly due to the significant biological heterogeneity within current diagnostic categories, which are based on symptoms alone. The authors propose that MTL, by learning multiple related tasks simultaneously, could leverage shared neuroimaging signatures and genetic information (like Copy Number Variants or CNVs) to improve diagnostic accuracy. They aim to explore if MTL can overcome the limitations of small sample sizes and improve diagnostic performance for psychiatric conditions and CNVs.
Methods
The study employs a Multi-Layer Perceptron (MLP) architecture for its MTL models, specifically a "shared bottom" approach where initial layers are shared across tasks, and then the network branches into task-specific heads. They also explore variations like MLPconn_deeper, MLPconcat, CNN, and SM models.
For predicting age and sex across different data collection sites, each site was treated as a separate task. For diagnosing psychiatric conditions and CNVs, each condition was treated as a task.
To establish a baseline, standard single-task learning models (MLP, SVC, LR, Ridge) were used to predict each condition independently.
Datasets
The study utilizes a large resting-state fMRI (rs-fMRI) dataset compiled from multiple studies, including:
- Clinical genetic datasets: Four cohorts focusing on individuals with CNVs associated with psychiatric disorders (Simons Variation in Individuals Project (SVIP), DEFINE Neuropsychiatric-CNVs Project, UCLA 22q11.2 CNV project, and the Montreal rare genomic disorder family project).
- Psychiatric conditions cohorts: Five datasets including individuals with Autism Spectrum Disorder (ASD), Attention-Deficit/Hyperactivity Disorder (ADHD), Schizophrenia (SZ), and Bipolar Disorder (BIP) (ABIDE1, ABIDE2, ADHD-200, CNP, and SZ studies).
- Unselected population: A large dataset from the UK Biobank, used for age and sex prediction experiments.
This dataset comprises a total of 2872 participants across 53 sites, covering 7 CNVs and 4 psychiatric disorders.
Key Insights and Ideas
- MTL for site harmonization is effective: When tasks are similar (predicting age/sex) and differ primarily due to data collection sites, MTL can indeed improve prediction accuracy, especially with larger sample sizes. This suggests MTL can help overcome site-specific biases.
- Sample size is paramount for MTL: The effectiveness of MTL is highly dependent on sample size. For age prediction, MTL was detrimental for small sample sizes (<500) but beneficial when sample sizes were large (>1500). This highlights that MTL's ability to pool data is only advantageous when there's enough data to begin with.
- MTL struggles with heterogeneous diagnostic tasks: Surprisingly, applying MTL to a combined set of psychiatric conditions and CNVs harmed diagnostic accuracy. This suggests that the inherent heterogeneity and potentially weak or non-existent shared biological factors across these diverse diagnostic categories prevent MTL from learning effectively.
- Task relationships are not stable: Pairwise MTL analysis revealed that the relationships between diagnostic tasks were not consistent across different model architectures. This implies that the "shared latent factors" hypothesis might be too simplistic for complex psychiatric conditions and CNVs, and that sample size and individual task performance are stronger drivers of MTL success.
- Thousands of patients needed for diagnostic MTL: The study's findings strongly suggest that developing a successful MTL model for diagnosing psychiatric conditions using fMRI would require datasets with thousands of patients across different diagnoses, far beyond what was available in this study.
Key Results
- Age/Sex Prediction (UK Biobank):
- For sex prediction, MTL across three UKBB sites showed gains over single-task learning, especially with larger sample sizes (N > 7500).
- For age prediction, MTL was beneficial only when increasing sample size through site pooling (N > 1500 total subjects). For smaller sample sizes, single-task and multi-task models performed similarly.
- When applied across a varied collection of sites, MTL improved sex prediction accuracy in most sites and significantly outperformed single-task learning for age prediction (mean loss reduced from 12.2 to 8.9 years²).
- Diagnostic Accuracy (CNVs & Psychiatric Conditions):
- Single-task ML models achieved reasonable diagnostic accuracy for some conditions (e.g., DEL 22q11.2 up to 90%, SZ, BIP, DEL 16p11.2, etc., over 70%).
- MTL applied to the combined set of 7 CNVs and 4 psychiatric conditions decreased overall prediction accuracy compared to single-task learning.
- Pairwise MTL analysis showed that conditions with high single-task accuracy but small sample sizes (e.g., DEL 22q11.2, DEL 16p11.2) suffered when trained with partners. Larger sample size conditions (SZ, ADHD, ASD) were less affected.
Limitations and Future Work
- Sample Size Limitation: The most significant limitation is the relatively small sample size for many of the CNV and psychiatric condition cohorts, which the authors argue is the primary reason MTL failed to improve diagnostic accuracy. They suggest that thousands of patients per condition are likely needed.
- Heterogeneity of Tasks: The authors acknowledge that the tasks (diagnoses) might be too heterogeneous to benefit from joint learning. Future work could explore approaches that focus on finding homogeneous subtypes within conditions.
- Data Harmonization: While MTL showed benefits for site harmonization, the paper notes that further research into data harmonization techniques is ongoing.
- Model Architectures: While they explored several MTL architectures, it's possible that other, more advanced architectures or different parameter-sharing schemes could yield better results.
- Future Directions: The authors suggest that future research should focus on increasing the amount of data available for ML in neuroimaging. They also hint at the potential of self-supervised or transfer learning approaches to overcome sample size limitations.
Summary updated