Deep Multiway Canonical Correlation Analysis For Multi-Subject Eeg Normalization

Jaswanth Reddy Katthi, Sriram Ganapathy

Abstract

Extracting stimulus-related signals from electroencephalography (EEG) data is challenging due to a low signal-to-noise ratio (SNR) and inter-subject variability. We present Deep Multiway Canonical Correlation Analysis (DMCCA), a deep learning framework that normalizes EEG data across subjects by maximizing inter-subject correlation while minimizing reconstruction error through an auto-encoder architecture. Evaluated on natural speech and music datasets, DMCCA significantly outperforms traditional linear methods, achieving up to a 0.29 increase in Pearson correlation and enhancing the extraction of shared neural responses to naturalistic stimuli.

Introduction

Electroencephalography (EEG) provides high temporal resolution for monitoring brain activity, yet stimulus-related signals are often obscured by significant noise, resulting in an average SNR of approximately -20 dB. Traditional methods rely on repeated trials of short stimuli to average out noise, a technique that is impractical for naturalistic stimuli like continuous speech or music. Furthermore, individual differences in brain anatomy and functional organization necessitate "normalization" to find common neural representations across subjects. While linear Multiway Canonical Correlation Analysis (MCCA) is a standard tool for this task, it often fails to capture the complex, non-linear relationships inherent in auditory processing.

Methods: Deep Multiway CCA (DMCCA)

The proposed DMCCA architecture extends multi-view learning by incorporating a deep auto-encoder structure. The model consists of three primary components:

  1. Shared Encoders: Each subject's EEG data ($x_n$) is processed by an individual neural network encoder ($f_n$) that projects high-dimensional data into a lower-dimensional common subspace.
  2. Correlation Maximization: The objective function maximizes the correlation between these projected representations across all $N$ subjects, forcing the network to isolate stimulus-driven activity from subject-specific noise.
  3. Reconstruction Regularization: Unlike previous deep CCA variants, DMCCA includes decoders that attempt to reconstruct the original EEG signals from the shared representation ($y$). This reconstruction task acts as a critical regularizer, preventing the model from overfitting to spurious correlations in the noisy data.
Comparison of the standard Deep Generalized CCA and the proposed DMCCA architecture featuring reconstruction decoders.

The joint loss function optimizes a weighted combination of inter-subject correlation and Mean Squared Error (MSE) of the reconstruction.

Experimental Results

The model was validated using two datasets: a speech dataset (6 subjects) and the NMED-H music dataset (48 subjects). DMCCA demonstrated superior performance over linear MCCA (LMCCA) across all metrics.

Correlation Improvements

DMCCA achieved substantial gains in Pearson correlation values, with an average improvement of 0.08 for speech and 0.29 for music. As shown in the subject-wise analysis, DMCCA consistently provided higher correlation scores than the linear baseline, indicating a more effective extraction of the shared stimulus response.

Subject-wise correlation comparison between Linear MCCA (LMCCA) and Deep Multiway CCA (DMCCA).

Classification and Robustness

In "match-vs-mismatch" classification tasks, DMCCA outperformed linear methods across various segment lengths (1s to 32s). For instance, on the NMED-H dataset with 4s segments, DMCCA reached significantly higher performance metrics compared to LMCCA. The inclusion of the decoder was found to be essential; without the reconstruction loss, the deep network tended to overfit the high-dimensional noise inherent in EEG recordings.

Discussion and Conclusion

The success of DMCCA highlights that the brain's response to complex, naturalistic audio is fundamentally non-linear. By leveraging deep learning and auto-encoder regularization, DMCCA effectively "normalizes" EEG data across subjects, significantly boosting the usable signal. While computationally more intensive than linear methods, the framework offers a robust path forward for improving brain-computer interfaces (BCIs) and auditory tracking technologies. Future research will focus on scaling the model to larger cohorts and improving the interpretability of the learned non-linear features.

Summary updated

BibTeX
@inproceedings{katthi2021deep,
  author = {Jaswanth Reddy Katthi and Sriram Ganapathy},
  title = {{Deep Multiway Canonical Correlation Analysis For Multi-Subject Eeg Normalization}},
  booktitle = {IEEE International Conference on Acoustics, Speech and Signal Processing},
  year = {2021},
  doi = {10.1109/icassp39728.2021.9414274},
}
Figure