Abstract
Traditional stimulus-based analysis methods of magnetoencephalography (MEG) data are often dissatisfactory when applied to naturalistic experiments where two or more subjects are measured either simultaneously or sequentially. To uncover the commonalities in the brain activity of the two subjects, we propose a method that searches for linear transformations that output maximally correlated signals between the two brains. Our method is based on canonical correlation analysis (CCA), which provides linear transformations, one for each subject, such that the temporal correlation between the transformed MEG signals is maximized. Here, we present a non-linear version of CCA which measures the correlation of energies and allows for a variable delay between the time series to accommodate, e.g., leader-follower changes. We test the method with simulations and with MEG data from subjects who received the same naturalistic stimulus sequence. The method may help analyse future experiments where the two subjects are measured simultaneously while engaged in social interaction.
Introduction: The Challenge of Hyperscanning
Studying how two brains "click" during social interaction—a field known as hyperscanning—is notoriously difficult. Traditional Magnetoencephalography (MEG) analysis relies on averaging data across many trials of the same stimulus to boost the signal-to-noise ratio (SNR). However, in naturalistic settings (like a conversation or watching a movie), brain responses aren't perfectly "phase-locked"; two people might process the same information at slightly different times or with different neural rhythms.
Standard linear methods like Canonical Correlation Analysis (CCA) struggle here because they look for direct temporal correlations between raw signals. If Subject A leads and Subject B follows with a slight, fluctuating delay, or if the commonality lies in the intensity of an oscillation rather than its exact phase, standard CCA fails. The authors aim to bridge this gap by looking for correlations in the power (energy) of brain signals while allowing for flexible time lags.
Methodology: Non-Linear CCA (NLCCA)
The core innovation is Non-Linear CCA (NLCCA). The method breaks down the complexity of neural synchronization through several key steps:
- Spatial Filtering: Instead of looking at individual sensors, the method calculates "spatial filters" (weighted combinations of sensors) for each subject to isolate specific neural sources.
- Energy Correlation: Rather than correlating the raw filtered signals, the algorithm correlates their squared amplitudes (energies). This captures shared "bursts" of activity even if the underlying waves aren't perfectly in sync.
- The "Binomial Trick": To make the non-linear math tractable, the authors expand the signal space into "binomials"—products of all possible pairs of signal components. This effectively turns a non-linear problem into a linear one in a higher-dimensional space, allowing for efficient iterative solvers.
- Dynamic Lag Estimation: The model includes a time-delay parameter ($\tau$) that can vary over the course of the experiment, which is crucial for social dynamics where "leadership" might shift between participants.
Experimental Validation
The authors validated NLCCA using two distinct approaches:
- Synthetic Data: They simulated two brain sources (occipital and parietal) with varying strengths and added real "brain noise" recorded from a resting subject. This allowed them to test if the algorithm could successfully separate two distinct, correlated sources.
- Naturalistic Stimulation Data: They used a dataset of 11 subjects who all watched the same 8-minute sequence of visual, auditory, and tactile stimuli. To test the algorithm's ability to find delays, they artificially introduced lags (0 to 1000 ms) between different subjects' data.
Key Results and Insights
The study yielded several critical insights into the performance of NLCCA:
- Accurate Lag Detection: In the real MEG data, NLCCA was remarkably accurate at recovering the artificially introduced time delays, with estimated lags closely following the theoretical "ground truth" ($r = 0.74$).
- Source Separation: In synthetic tests where one source was 30% the strength of the other, the method successfully separated the two sources into distinct components with high correlation coefficients (~0.99).
- Energy is Key: In naturalistic data, the envelope of neural activity is often more consistent across people than the fast-changing raw phase. Correlating energies makes the analysis much more robust to the "jitter" of real-world timing.
- Strength in Numbers: While spatial filters for individual pairs were noisy, concatenating data from multiple subjects allowed the algorithm to identify clear, consistent spatial patterns across the groups.
Limitations and Future Directions
The authors acknowledge a few hurdles for real-world application. Estimating a time-varying delay is computationally expensive, especially if high temporal precision is needed to track rapid social shifts. Furthermore, the method currently requires significant data to produce stable, reliable spatial maps for individual pairs of subjects.
The next major step is applying NLCCA to true live social interaction, where the delays and correlations are emergent rather than pre-programmed. This will move the field from studying "pseudo-interaction" to understanding the dynamic neural coupling that occurs during real-world social engagement.
Summary updated