TL;DR
This paper introduces a novel Sparse Partial Least Squares (SPLS) framework that enhances model reliability by testing its performance across multiple random data splits. This approach successfully identified two significant associations between brain structure and individual cognitive tasks in dementia patients, outperforming traditional Partial Least Squares (PLS) and demonstrating the superiority of a new "projection deflation" method for SPLS.
What Problem Does This Paper Address?
Studying complex neurological and psychiatric diseases like dementia often involves analyzing high-dimensional neuroimaging data alongside clinical assessments. Traditional supervised machine learning methods can struggle when diagnostic labels are unreliable or when the goal is to understand the intricate relationships between brain structure and specific cognitive functions, rather than just classifying patients. Exploratory methods like Partial Least Squares (PLS) can reveal these associations, but standard PLS produces complex, non-sparse models that are difficult to interpret. Sparse PLS (SPLS) addresses this by selecting important features, but existing SPLS methods can be computationally demanding and may not always yield stable results, especially when dealing with noisy clinical data like individual test items. Furthermore, the deflation methods used to extract sequential associative effects in SPLS might not be optimal for sparse models.
Methods
The core contribution is a new multiple hold-out framework for SPLS. This framework aims to improve the reliability and robustness of SPLS models. Key methodological innovations include:
- Hyper-parameter Optimization via Random Subsampling: Instead of traditional k-fold cross-validation, the framework uses repeated random splits of the data into training (80%) and testing (20%) sets for hyper-parameter tuning. This is repeated 100 times for each combination of sparsity parameters (Cu, Cv) to estimate average correlations and select the optimal settings.
- Statistical Evaluation using Hold-out Datasets: To avoid the computational burden of re-optimizing hyper-parameters for every permutation in statistical testing, the framework uses a fixed hold-out dataset. Permutation tests are performed on the remaining data, and the resulting p-values are calculated based on the correlation with the hold-out data. This process is repeated 9 times with different hold-out sets, and an "omnibus hypothesis" is used to determine overall statistical significance.
- Projection Deflation: A novel deflation strategy is introduced and compared to the standard Hotelling's deflation (used in PLS). Projection deflation aims to enforce orthogonality between successive weight vector pairs by projecting the data matrices onto the space spanned by the current weight vectors and subtracting this projection. This is hypothesized to lead to more distinct and reliable associative effects.
Datasets
The study utilized data from the Alzheimer's Disease Neuroimaging Initiative (ADNI) database. This dataset comprised:
- Neuroimaging Data: T1-weighted MRI scans from 592 subjects (diagnosed as healthy, mild cognitive impairment, or Alzheimer's disease). These scans were processed to generate grey matter probability maps, normalized, and masked to select relevant voxels.
- Clinical Data: Individual item scores from the Mini-Mental State Examination (MMSE). The MMSE is a widely used cognitive assessment tool, and this study treated each question/task as a separate variable, rather than using a summary score.
Key Insights and Ideas
- Individual MMSE Items Hold Valuable Information: The study demonstrates that analyzing individual MMSE questions/tasks, despite their inherent noise, can reveal subtle cognitive associations with brain structure that might be missed by using only summary scores.
- Sparsity is Crucial for Interpretability: SPLS, by enforcing sparsity, successfully identified specific subsets of brain voxels and MMSE items that are associated, making the findings much more interpretable than standard PLS.
- Projection Deflation is Superior for SPLS: The novel "projection deflation" method proved more effective than traditional PLS deflation for SPLS, leading to statistically significant results and better generalization to unseen data. This suggests that deflation strategies should be tailored to the specific properties of sparse methods.
- Robustness Through Multiple Hold-outs: The multiple hold-out framework significantly increases the reliability of the statistical findings by reducing the dependence on any single data split.
Key Results
- Two Statistically Significant Associative Effects: The SPLS framework, using projection deflation, identified two significant associations between subsets of brain voxels and subsets of MMSE items.
- PLS Failed to Find Significant Effects: Standard PLS, even with the proposed framework, did not yield any statistically significant associative effects.
- Projection Deflation Outperforms PLS Deflation: SPLS with projection deflation found a second statistically significant associative effect, whereas PLS deflation did not. Projection deflation also resulted in higher average correlations on hold-out datasets, indicating better generalizability.
- Improved Generalizability: The SPLS models, particularly with projection deflation, showed better performance on unseen data, as evidenced by higher average absolute correlations on hold-out datasets and smaller standard deviations.
- Continuous Neurodegeneration Spectrum: Projection of subjects onto the SPLS latent space revealed a continuous distribution from healthier to more degenerated states, rather than distinct clusters based on clinical diagnosis.
Limitations and Future Work
- Strict Statistical Thresholding: The authors acknowledge that the statistical evaluation step, using multiple hold-out sets and Bonferroni correction, is quite strict and might lead to false negatives, potentially requiring large datasets.
- Application to Other Domains: The framework is suggested for future application to other areas where clinical categories have limitations, such as psychiatric disorders, to aid in patient stratification and understanding.
- Further Exploration of Deflation: While projection deflation showed promise, further research into optimal deflation strategies for sparse methods could be beneficial.
Summary updated