TL;DR
This paper introduces "Hybrid Harmony," an open-source software application for multi-person neurofeedback that enables real-time measurement and visualization of inter-brain synchrony. It aims to bridge the gap between hyperscanning research and practical applications by providing a flexible platform for exploring how synchrony can be leveraged to enhance social interactions and understand interpersonal dynamics.
What Problem Does This Paper Address?
Understanding and quantifying "being on the same wavelength" during social interactions is a complex challenge. While hyperscanning techniques allow researchers to measure synchronized brain activity across multiple individuals, translating this into practical tools for improving social connection or understanding interpersonal dynamics has been limited. Existing multi-person neurofeedback systems are scarce, and there's a lack of consensus on the best metrics for quantifying inter-brain synchrony. This paper addresses the need for a versatile and accessible platform that can facilitate research into the effects of synchrony neurofeedback and its potential applications.
Methods
The core of this work is the Hybrid Harmony software, a Brain-Computer Interface (BCI) application built using Python. It supports simultaneous EEG data acquisition from multiple participants via LabStreamingLayer (LSL). The software offers a flexible pipeline for:
- Data Preprocessing: Buffering and segmenting incoming EEG streams.
- Connectivity Analysis: Computing various inter-brain synchrony metrics in real-time. Users can select from a range of metrics, including coherence, imaginary coherence, envelope correlation, power correlation, phase-locking value (PLV), and Circular Correlation Coefficient (CCorr). The software allows for different "connectivity types" (e.g., one-to-one, all-to-all) and frequency band selection.
- Normalization: Implementing manual or file-based baselining for connectivity values.
- Visualization and Sonification: Outputting synchrony data via LSL and Open Sound Control (OSC) to external modules. The paper describes a visualization module that translates synchrony into visual cues (e.g., distance between avatars) and a sonification module that can modulate audio parameters based on synchrony levels.
The novelty lies in the integration of multiple connectivity metrics within a real-time neurofeedback framework, coupled with flexible visualization and sonification capabilities, all within an open-source platform.
Datasets
The paper describes a validation study using a dataset of 236 dyads who participated in the "Mutual Wave Machine" experiment. This dataset was used to assess whether Hybrid Harmony's output correlates with socially relevant self-report measures. No new benchmarks were introduced; rather, existing experimental data was utilized.
Key Insights and Ideas
- Bridging the Gap: Hybrid Harmony aims to make hyperscanning neurofeedback more accessible for researchers and developers, moving beyond purely experimental setups.
- Flexibility is Key: The ability to choose and switch between various synchrony metrics is crucial, given the lack of consensus on which metric best reflects specific social phenomena.
- Socially Relevant Outputs: The study demonstrates that inter-brain synchrony, as measured by Hybrid Harmony, can indeed capture and correlate with important interpersonal characteristics like social closeness and personal distress. This suggests that synchrony neurofeedback could be a viable tool for interventions targeting these aspects.
- Artistic and Game Development Potential: The software's visualization and sonification capabilities open doors for creative applications in biofeedback art and serious games, allowing for novel forms of interactive multiplayer experiences.
Key Results
The validation study with 236 dyads showed:
- Negative Correlation with Personal Distress: Pairs' average personal distress was negatively correlated with their neurofeedback synchrony in theta, alpha, and beta bands.
- Positive Correlation with Social Closeness: Social closeness was positively correlated with pairs' neurofeedback synchrony in alpha and beta bands.
These results, while not proving the efficacy of Hybrid Harmony as a therapeutic tool, importantly demonstrate that its real-time output is sensitive to socially relevant psychological features.
Limitations and Future Work
- Data Fidelity and Artifacts: The paper acknowledges the challenge of motion artifacts in EEG data, which can compromise synchrony measures. While they opted for a transparency approach with participants, future work will incorporate online data cleaning procedures and support for systems with built-in cleaning options.
- Real-time Interpretation: The real-time nature of the analysis, while necessary for immediate feedback, might limit the characterization of synchrony that occurs over longer or non-aligned temporal intervals.
- Metric Correspondence: There's still limited understanding of how different synchrony metrics map to specific psychological processes and social contexts. Future work will aim to provide more guidance on metric selection.
- Expanding Modalities: The authors express a desire to extend Hybrid Harmony to incorporate other data streams, such as physiological signals (heart rate, respiration) and movement data, to provide a more holistic view of interpersonal synchrony.
- fNIRS Integration: Given the growing fNIRS hyperscanning literature, extending Hybrid Harmony to support fNIRS data is a significant future direction.
- Adaptive Normalization: Introducing temporal delays and adaptive normalization could enhance sensitivity to endogenous synchronizers and allow for more controlled experimental designs.
Summary updated