Spectral Introspection Identifies Group Training Dynamics in Deep Neural Networks for Neuroimaging

Bradley T. Baker, Vince D. Calhoun, Sergey M. PlisarXiv 2024 · DOI · OpenReview

Introduction

Deep Neural Networks (DNNs) have become indispensable tools for analyzing complex neuroimaging data. However, their "black box" nature remains a significant barrier to clinical adoption. Existing interpretability methods are primarily post-hoc, analyzing static, fully trained models. These methods often fail to capture the dynamic process of learning and can produce inconsistent saliency maps. To address these limitations, we introduce AutoSpec, an introspection framework that utilizes Singular Value Decomposition (SVD) to monitor the mathematical spectrum of gradients during the training process.

Methodology: The AutoSpec Framework

AutoSpec leverages the fundamental mechanics of backpropagation. In reverse-mode auto-differentiation, the gradient of a weight matrix ($\nabla W$) is decomposed into the product of input activations ($A$) and error signals (adjoint variables, $\Delta$). By performing SVD on these components in real-time, AutoSpec tracks the "spectrum" of singular values.

This approach allows for:

  1. Rank Monitoring: Quantifying the effective dimensionality of information utilized at each layer.
  2. Gradient Decomposition: Calculating group-specific gradients (e.g., patients vs. controls) to observe how different data cohorts drive the learning process without interrupting training.

Experimental Setup

The framework was validated across a wide range of data modalities and architectures, as summarized in the table below.

Table 1: Overview of the experimental datasets, modalities, and model architectures used to validate AutoSpec.

The study included baseline numerical data (MNIST), tabular neuroimaging (FreeSurfer), and functional neuroimaging (COBRE) for schizophrenia research.

Results and Discussion

Task-Specific Spectral Fingerprints

Our analysis revealed that gradient spectra serve as unique "fingerprints" for specific tasks. For instance, Autoencoders exhibit high-rank dynamics in early layers as they attempt to reconstruct complex inputs. In contrast, Classifiers quickly compress information, narrowing their spectral focus to task-relevant features.

Training Trajectories and Group Dynamics

Spectral plots illustrate how singular values evolve over time. In models like BERT and LSTMs, we observed a consistent trend of "spectral shrinking," indicating information compression as the model converges.

Figure 1: Spectral trajectories of singular values over 1,000 epochs. Panel (a) shows an MLP Autoencoder with ReLU activations, highlighting the evolution of internal representations.

Crucially, AutoSpec identified significant differences in training dynamics between schizophrenia patients (SZ) and healthy controls (HC). Statistical tests on the gradient spectra pinpointed specific layers and training epochs where the model's perception of these groups diverged, suggesting that the model captures distinct features for different clinical populations during learning.

Conclusion and Future Work

AutoSpec provides a novel window into the training dynamics of DNNs, offering insights that post-hoc methods miss. While the computational cost of real-time SVD is a current limitation, future work will focus on analytic spectral updates and scaling the framework to larger architectures like foundation models for medical imaging. By identifying group-specific biases during training, AutoSpec paves the way for more transparent and equitable neuroimaging AI.

Summary updated

Abstract

Neural networks, whice have had a profound effect on how researchers study complex phenomena, do so through a complex, nonlinear mathematical structure which can be difficult for human researchers to interpret. This obstacle can be especially salient when researchers want to better understand the emergence of particular model behaviors such as bias, overfitting, overparametrization, and more. In Neuroimaging, the understanding of how such phenomena emerge is fundamental to preventing and informing users of the potential risks involved in practice. In this work, we present a novel introspection framework for Deep Learning on Neuroimaging data, which exploits the natural structure of gradient computations via the singular value decomposition of gradient components during reverse-mode auto-differentiation. Unlike post-hoc introspection techniques, which require fully-trained models for evaluation, our method allows for the study of training dynamics on the fly, and even more interestingly, allow for the decomposition of gradients based on which samples belong to particular groups of interest. We demonstrate how the gradient spectra for several common deep learning models differ between schizophrenia and control participants from the COBRE study, and illustrate how these trajectories may reveal specific training dynamics helpful for further analysis.
BibTeX
@misc{baker2024spectral,
  author = {Bradley T. Baker and Vince D. Calhoun and Sergey M. Plis},
  title = {{Spectral Introspection Identifies Group Training Dynamics in Deep Neural Networks for Neuroimaging}},
  year = {2024},
  eprint = {2406.11825},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2406.11825},
}
Figure