Reconstructing subject-specific effect maps

Ender Konukoglu, Ben GlockerNeuroImage 2018 · DOI · OpenReview

TL;DR

This paper introduces RSM, a novel method to reconstruct cleaner, more reliable subject-specific "effect maps" from neuroimaging data. By leveraging a Bayesian approach with a prior model, RSM significantly reduces noise and improves the accuracy of detecting localized disease effects, making it a valuable tool for clinical and neuroscience research.

What Problem Does This Paper Address?

Neuroimaging studies often aim to identify how diseases or conditions affect the brain. While "global" inference (e.g., diagnosing a condition) is common, "local" inference – pinpointing specific areas of change within an individual's brain – is more challenging. Existing methods for creating these "subject-specific effect maps" often produce noisy results with scattered, isolated detections, making them difficult to interpret and use reliably. This noise stems primarily from sampling errors when training predictive models on limited datasets. The authors aim to address this by developing a method that can reconstruct cleaner, more accurate subject-specific effect maps, even in a diagnostic setting where the condition's presence is unknown.

Methods

The core of the proposed method, RSM (Reconstruction Subject-specific Effect Maps), is a Maximum-A-Posteriori (MAP) estimation framework. It treats the noisy subject-specific effect maps produced by standard binary classifiers as observations of an underlying "true" effect map.

Key components of RSM include:

  • Observation Model: This models the relationship between the observed noisy effect map ($\hat{p}$) and the true underlying effect map ($p$), incorporating measurement-dependent noise variance ($\sigma_j^2$).
  • Prior Model: A Markov Random Field (MRF) is used as the prior. This prior has two terms:
  • Unary term: Addresses unreliable measurements that don't consistently show a condition effect across the population, penalizing deviations from zero for such measurements.
  • Pairwise term: Enforces spatial consistency by encouraging neighboring measurements to have similar effect values, leveraging the anatomical correlation in neuroimaging data. This term can be either non-stationary (site-dependent variance, $\theta_{jk}$) or stationary (a single variance value).
  • Parameter Estimation: The parameters of the prior model and noise variance are estimated from the training data using bootstrap sampling and cross-validation.
  • Thresholding: After obtaining the reconstructed continuous effect map ($p^*$), a thresholding procedure is applied to generate binary maps ($q$). This threshold is determined by limiting the False Positive Rate (FPR) on control subjects, ensuring a conservative approach.

RSM is designed as a wrapper algorithm, meaning it can be used with various existing binary classifiers (e.g., ew-GMM, SVM, Logistic Regression).

Datasets

  • Synthetic Dataset: A custom dataset was generated with 200 samples (100 control, 100 case) and known ground truth for effect maps. This allowed for quantitative evaluation of detection accuracy.
  • Alzheimer's Disease Neuroimaging Initiative (ADNI) Database: This real-world dataset, containing structural T1-weighted MRI scans (specifically cortical thickness maps) from 290 subjects (145 AD patients, 145 age/gender-matched controls), was used to evaluate the method's performance on a clinical application. Longitudinal data (scans taken 6 months apart) was also used to assess reliability.

Key Insights and Ideas

  • Noise is a Major Hurdle: The paper strongly emphasizes that sampling error from finite training sets is a primary source of noise in subject-specific effect maps, leading to fragmented and unreliable detections.
  • Bayesian Reconstruction for Cleaner Maps: By framing the problem as a MAP estimation with a spatially aware prior, RSM effectively "denoises" the initial classifier outputs, producing more contiguous and interpretable maps.
  • Diagnostic Capability is Crucial: The ability to generate these maps without prior knowledge of the subject's condition (diagnostic manner) is highlighted as a key advantage for clinical applications.
  • Non-stationary Prior is Beneficial: The non-stationary pairwise term, which accounts for varying consistency across different brain regions, generally outperforms the stationary version, especially for higher regularization parameters. This suggests that anatomical heterogeneity is important to consider.
  • Classifier Choice Matters: While RSM improves results for most classifiers, its effectiveness can be influenced by the classifier's underlying assumptions (e.g., Gaussian prior in RSM vs. Laplace prior in L1-regularized LR).

Key Results

  • Synthetic Data: RSM consistently improved detection accuracy compared to using classifiers directly or with naive bootstrap averaging across various effect sizes and classifiers. For example, Dice Similarity Coefficients (DSC) were substantially improved.
  • ADNI Dataset:
    • RSM improved the correlation between subject-specific detections and auxiliary AD markers (MMSE, CSF a-β) for ew-GMM, SVM, and LR L2.
  • Reliability of subject-specific detections (measured by DSC between scans taken 6 months apart) was significantly enhanced by RSM, particularly for AD subjects.
  • Visual results showed that RSM produced fewer but larger contiguous affected areas compared to bootstrap averaging, leading to cleaner maps.
  • Outlier Detection: While used as a benchmark, outlier detection methods showed lower accuracy and higher false positives, highlighting the need for condition-specific approaches.
  • LR L1 Performance: The L1-regularized Logistic Regression classifier showed weaker performance, both with and without RSM, suggesting potential limitations of L1 regularization for this task.

Limitations and Future Work

  • Low Effect Sizes: The method, like most statistical inference methods, struggles with detecting very small effect sizes. Improving sensitivity for such cases is an area for future research.
  • Measurement Error: The current model assumes negligible measurement error. For modalities with significant inherent noise (e.g., functional connectivity), the $\sigma_j$ estimation might be underestimated, leading to higher false positives.
  • Assumption on Controls: The thresholding relies on the assumption that control subjects have no condition effects, which is a conservative approach. Relaxing this assumption (e.g., assuming a percentage of controls might have effects) could be explored.
  • Registration Dependency: RSM relies on spatially normalized data, requiring registration algorithms. Future work could explore registration-free methods.
  • Classifier Extensions: The current work focuses on linear binary classifiers. Extending RSM to multi-label classification and regression is a promising avenue.
  • Parameter Estimation for Complex Models: Applying RSM to methods with computationally expensive training might require developing alternative parameter estimation strategies beyond bootstrap sampling.
  • Integration with Biophysical Models: Integrating subject-specific detections with biophysical mechanistic models of disease progression could offer deeper insights.

Summary updated

Abstract

Predictive models allow subject-specific inference when analyzing disease related alterations in neuroimaging data. Given a subject's data, inference can be made at two levels: global, i.e. identifiying condition presence for the subject, and local, i.e. detecting condition effect on each individual measurement extracted from the subject's data. While global inference is widely used, local inference, which can be used to form subject-specific effect maps, is rarely used because existing models often yield noisy detections composed of dispersed isolated islands. In this article, we propose a reconstruction method, named RSM, to improve subject-specific detections of predictive modeling approaches and in particular, binary classifiers. RSM specifically aims to reduce noise due to sampling error associated with using a finite sample of examples to train classifiers. The proposed method is a wrapper-type algorithm that can be used with different binary classifiers in a diagnostic manner, i.e. without information on condition presence. Reconstruction is posed as a Maximum-A-Posteriori problem with a prior model whose parameters are estimated from training data in a classifier-specific fashion. Experimental evaluation is performed on synthetically generated data and data from the Alzheimer's Disease Neuroimaging Initiative (ADNI) database. Results on synthetic data demonstrate that using RSM yields higher detection accuracy compared to using models directly or with bootstrap averaging. Analyses on the ADNI dataset show that RSM can also improve correlation between subject-specific detections in cortical thickness data and non-imaging markers of Alzheimer's Disease (AD), such as the Mini Mental State Examination Score and Cerebrospinal Fluid amyloid-β levels. Further reliability studies on the longitudinal ADNI dataset show improvement on detection reliability when RSM is used.
BibTeX
@article{konukoglu2018reconstructing,
  author = {Ender Konukoglu and Ben Glocker},
  title = {{Reconstructing subject-specific effect maps}},
  journal = {NeuroImage},
  year = {2018},
  doi = {10.1016/j.neuroimage.2018.07.032},
}
Figure