Identifying Alcohol Use Disorder With Resting State Functional Magnetic Resonance Imaging Data: A Comparison Among Machine Learning Classifiers

Victor M. Vergara, Flor A. Espinoza, Vince D. CalhounFrontiers in Psychology 2022 · DOI

TL;DR

This study investigates the use of machine learning (ML) classifiers to identify Alcohol Use Disorder (AUD) using resting-state functional magnetic resonance imaging (rsFNC) data. By carefully controlling for comorbid substance use (specifically nicotine), the researchers found that a neural network classifier achieved the highest performance (AUC of 0.79), highlighting the potential of rsFNC as a biomarker for AUD.

What Problem Does This Paper Address?

Alcohol Use Disorder (AUD) is a significant public health issue with complex effects on the brain. Accurately detecting AUD and its neurological underpinnings is challenging, especially given that AUD often co-occurs with other substance use disorders, like nicotine dependence. Previous studies using neuroimaging data to identify AUD have often included participants with comorbid substance use, making it difficult to isolate the specific effects of alcohol. This paper aims to address this by investigating the efficacy of various ML classifiers in detecting AUD using rsFNC data from individuals free of nicotine and other substance dependencies, thereby providing a cleaner signal for alcohol's impact.

Methods

The researchers employed a rigorous machine learning pipeline. First, resting-state functional magnetic resonance imaging (rsFNC) data was preprocessed and analyzed using independent component analysis (ICA) to extract functional networks. From these networks, functional connectivity estimates between all pairs of regions were calculated, resulting in a large feature set.

A crucial step was feature selection using a random forest algorithm. This method not only helps identify the most informative connections for classification but also provides a measure of "feature importance." The study explored different levels of feature selection, from the top 10% to 100% of features.

Ten different machine learning classifiers were then evaluated: Random Forest, Logistic Regression, Nearest Neighbors, Support Vector Machine (Linear and RBF kernels), Gaussian Process, Decision Tree, Neural Network (multilayer perceptron), AdaBoost, Naive Bayes, and Quadratic Discriminant Analysis (QDA).

A 10-fold cross-validation strategy was implemented to ensure robust evaluation and prevent data leakage. This means the data was split into 10 parts, with each part used once as a testing set while the remaining nine were used for training. Model tuning was performed using a grid search to optimize hyperparameters for each classifier. The primary metric for evaluating classifier performance was the Area Under the Curve (AUC), which quantifies a classifier's ability to distinguish between AUD and control groups.

Datasets

The study utilized data from a cohort scanned at The Mind Research Network in Albuquerque, New Mexico. The dataset comprised:

  • 51 individuals diagnosed with Alcohol Use Disorder (AUD).
  • 51 age- and sex-matched control subjects (non-AUD).

A key characteristic of this dataset is that all 102 participants were free of nicotine, cannabis, or any other dependence or habit-forming substance use. This strict exclusion criterion is a significant differentiator from many previous studies in this area. The AUD status was determined using the Alcohol Use Disorder Identification Test (AUDIT) and the Structured Clinical Interview for DSM-IV-TR Axis I Disorders (SCID-I/P) for controls.

Key Insights and Ideas

  • Isolating Alcohol's Impact: By meticulously excluding participants with comorbid nicotine use, this study provides a clearer picture of how alcohol itself affects brain connectivity, a critical step in understanding AUD's neurobiological basis.
  • rsFNC as a Viable Biomarker: The findings strongly support the utility of resting-state functional network connectivity (rsFNC) as a potential biomarker for AUD, even when analyzed in isolation from other substance use.
  • Neural Networks Shine: Among the tested classifiers, the neural network demonstrated the highest classification performance, suggesting its suitability for complex pattern recognition in neuroimaging data for AUD detection.
  • Feature Importance Reveals Key Networks: The feature selection process highlighted specific functional connections as most relevant for AUD classification, pointing towards the visual, sensorimotor, executive control, reward, and salience networks. This offers valuable insights into the brain regions most affected by AUD.
  • Nicotine's Potential Confounding Effect: The study implicitly suggests that previous findings might have been influenced by comorbid nicotine use, as the cleaner dataset yielded comparable or even improved results for some classifiers.

Key Results

  • The neural network classifier achieved the highest Area Under the Curve (AUC) of 0.79, indicating good discriminatory power between AUD and control groups.
  • Other strong performers included Logistic Regression, Nearest Neighbors, and Support Vector Machine (SVM) classifiers, all achieving similar AUC scores.
  • The worst performers were Gaussian Process and Quadratic Discriminant Analysis (QDA) classifiers, with AUC scores close to chance (around 0.50).
  • Feature selection using random forest identified functional connections between visual, sensorimotor, executive control, reward, and salience networks as most relevant for classification. Specifically, hypoconnectivity in visual and sensorimotor domains, and connections between the reward network (RN) and executive control network (ECN), and salience and language networks were highlighted.

Limitations and Future Work

  • Sample Size: While the study carefully selected its sample, the total number of participants (102) is relatively small for training machine learning models, which often benefit from larger datasets.
  • AUD Stratification: The study did not further stratify AUD into different severity levels, which could potentially reveal more nuanced patterns.
  • Withdrawal Period: The AUD participants were scanned within 24 hours of sobriety. The authors note that longer abstinence periods could influence the observed brain network patterns, and this difference in withdrawal time compared to other studies might affect comparability.
  • Comorbidity: The study's strength lies in its exclusion of comorbidities, but this also means it doesn't address how these ML models would perform in the presence of multiple substance use disorders.
  • Future Work: The authors suggest future research should explore ML models for multi-label classification to handle comorbidities. They also propose investigating the impact of different withdrawal periods and larger sample sizes to further validate and refine these findings for clinical applications.

Summary updated

Abstract

Alcohol use disorder (AUD) is a burden to society creating social and health problems. Detection of AUD and its effects on the brain are difficult to assess. This problem is enhanced by the comorbid use of other substances such as nicotine that has been present in previous studies. Recent machine learning algorithms have raised the attention of researchers as a useful tool in studying and detecting AUD. This work uses AUD and controls samples free of any other substance use to assess the performance of a set of commonly used machine learning classifiers detecting AUD from resting state functional network connectivity (rsFNC) derived from independent component analysis. The cohort used included 51 alcohol dependent subjects and 51 control subjects. Despite alcohol, none of the 102 subjects reported use of nicotine, cannabis or any other dependence or habit formation substance. Classification features consisted of whole brain rsFNC estimates undergoing a feature selection process using a random forest approach. Features were then fed to 10 different machine learning classifiers to be evaluated based on their classification performance. A neural network classifier showed the highest performance with an area under the curve (AUC) of 0.79. Other good performers with similar AUC scores were logistic regression, nearest neighbor, and support vector machine classifiers. The worst results were obtained with Gaussian process and quadratic discriminant analysis. The feature selection outcome pointed to functional connections between visual, sensorimotor, executive control, reward, and salience networks as the most relevant for classification. We conclude that AUD can be identified using machine learning classifiers in the absence of nicotine comorbidity.
BibTeX
@inproceedings{vergara2022identifying,
  author = {Victor M. Vergara and Flor A. Espinoza and Vince D. Calhoun},
  title = {{Identifying Alcohol Use Disorder With Resting State Functional Magnetic Resonance Imaging Data: A Comparison Among Machine Learning Classifiers}},
  booktitle = {Frontiers in Psychology},
  year = {2022},
  doi = {10.3389/fpsyg.2022.867067},
}
Figure