DN3: An open-source Python library for large-scale raw neurophysiology data assimilation for more flexible and standardized deep learning

Demetres Kostas, Frank RudziczbioRxiv 2020 · DOI

Abstract

Deep learning (DL) has transformed fields like computer vision, yet its application in neurophysiology—specifically electroencephalography (EEG)—remains hindered by fragmented datasets and non-standardized pipelines. We present DN3 (Deep Neural Networks for Neurophysiology), an open-source Python library designed to accelerate DL research by facilitating rapid, reproducible experimentation across multiple datasets. To demonstrate the utility of DN3, we introduce T-Vectors: task-independent EEG embeddings that uniquely identify individuals. Our results show that T-Vectors can achieve up to 97.7% accuracy in subject identification using as little as one second of raw data, generalizing effectively even to subjects unseen during training.

Introduction

The "reproducibility crisis" in Brain-Computer Interface (BCI) research often stems from a reliance on small, isolated datasets and idiosyncratic software implementations. While tools like MNE-Python provide robust signal processing capabilities, they are not natively optimized for the high-throughput training loops required by modern deep learning. DN3 aims to bridge this gap by providing a flexible, "middle-ground" framework that standardizes data assimilation while remaining compatible with expert-level DL experimentation.

The DN3 Library: Architecture and Data Assimilation

DN3 is built upon two core pillars: a human-readable configuration system and a standardized data hierarchy.

The Configuratron and Pipeline

At the heart of DN3 is the "Configuratron," a YAML-based tool that allows researchers to define dataset parameters, channel mappings, and preprocessing steps in a shareable format. This ensures that experiments can be replicated across different labs with minimal friction. The overall architecture separates the concerns of data loading and preprocessing from the actual model training process.

High-level overview of the DN3 architecture, illustrating the separation between data assimilation and the training process.

Standardized Data Hierarchy

To handle the complexities of multi-subject, multi-session recordings, DN3 organizes data into a strict hierarchy: Dataset → Thinker (Subject) → Recording (Session). This structure, illustrated in Figure 2, allows for the seamless implementation of validation strategies like "leave-one-subject-out" cross-validation, which are critical for assessing the generalizability of neural networks.

The hierarchical structure of a DN3 Dataset, showing the integration of MNE objects into Thinker and Recording layers.

Furthermore, DN3 introduces the "Deep1010" mapping system. By mapping 77 EEG channels to consistent tensor indices, DN3 enables models to be trained on heterogeneous datasets with varying electrode layouts without requiring manual re-coding for each new hardware configuration.

T-Vectors: A Case Study in EEG Biometrics

To validate the DN3 framework, we developed T-Vectors, a novel method for person disambiguation inspired by "X-Vectors" in speech recognition. Using the Temple University Hospital EEG Corpus (TUEG), we trained a deep neural network to classify nearly 1,000 individuals. By extracting activations from the model's late hidden layers, we produced fixed-length numerical summaries—T-Vectors—that represent a person's unique "brain signature."

Task Invariance and Generalization

A key finding of this research is "thinker invariance." T-Vectors generated during different mental tasks (e.g., motor imagery) remain remarkably consistent for a given individual. This suggests that the model captures intrinsic physiological traits rather than transient cognitive states. Furthermore, the system generalizes to unseen subjects; by applying a k-nearest neighbor approach to the T-Vector manifold, new users can be identified with high precision.

Results

The effectiveness of the DN3-enabled T-Vector approach was tested on two prominent motor imagery datasets: BCI Competition IV (BCIC) 2a and the PhysioNet Motor Imagery (MMI) database.

  • High Accuracy Biometrics: Subject identification reached 96.1% on BCIC and 97.7% on MMI using a simple k-nearest neighbor (k=5) approach.
  • Temporal Stability: On the BCIC dataset, where recordings were separated by several days, T-Vectors remained stable, demonstrating the robustness of the biometric signature over time.
  • Efficiency: High-confidence identification was achieved using sequences as short as 1 second, highlighting the potential for real-world biometric applications.

Conclusion and Future Work

DN3 provides a much-needed standardized infrastructure for neurophysiology research, enabling the development of powerful cross-dataset representations like T-Vectors. While these embeddings show great promise for biometrics, future research must address potential demographic biases and explore their utility as biomarkers for clinical diagnostics. By open-sourcing the DN3 library and the T-Vectors codebase, we hope to foster a more collaborative and reproducible ecosystem for deep learning in neuroscience.

Check out the project: The source code for DN3 and the T-Vectors project is available on GitHub.

Summary updated

Abstract

Abstract We propose an open-source Python library, called DN3, designed to accelerate deep learning (DL) analysis with encephalographic data. This library focuses on making experimentation rapid and reproducible and facilitates the integration of both public and private datasets. Furthermore, DN3 is designed in the interest of validating DL processes that include, but are not limited to, classification and regression across many datasets to prove capacity for generalization. We explore the effectiveness of this library by presenting a general scheme for person disambiguation called T-Vectors inspired by speech recognition. These are single vectors created by typically short, though arbitrary in length, electro-encephalographic (EEG) data sequences that uniquely identify users relative to others. T-Vectors were trained by classifying nearly 1000 people using as little as 1 second-long sequences and generalize effectively to users never seen during training. Generalized performance is demonstrated on two commonly used and publicly accessible motor imagery task datasets, which are notorious for intra- and inter-subject signal variability. According to these datasets, subjects can be identified with accuracies as high as 97.7% by simply adopting the label of the nearest neighbouring T-Vectors, with no dependence on task performed and little dependence on recording session, even when sessions are separated by days. Visualization of the T-Vectors from both datasets show no conflation of subjects between datasets, and indicates a T-Vector manifold where subjects cluster well. We first conclude that this is a desirable paradigm shift in EEG-based biometrics and secondly that this manifold deserves further investigation. Our proposed library provides a variety of essential tools that facilitated the development of T-Vectors. The T-vectors codebase serves as a template for future projects using DN3, and we encourage leveraging our provided model for future work. Author summary We present a new Python library to train deep learning (DL) models with brain data. This library is tailored, but not limited, to developing neural networks for brain-computer-interfaces (BCI) applications. There is abundant interest in leveraging DL in the wider neuroscience community, but we have found current solutions limiting. Furthermore both BCI and DL benefit from benchmarking against multiple datasets and sharing parameters. Our library tries to be accessible to DL novices, yet not limiting to experts, while making experiment configurations more easily shareable and flexible for benchmarking. We demonstrated many of the features of our library by developing a deep neural network capable of disambiguating people from arbitrary lengths of electroencephalography data. We identify a variety of future avenues of study for these representations produced by our network, particularly in biometric applications and addressing the variation in BCI classifier performance. We share our model, library and its associated guides and documentation with the community at large.
BibTeX
@misc{kostas2020dn,
  author = {Demetres Kostas and Frank Rudzicz},
  title = {{DN3: An open-source Python library for large-scale raw neurophysiology data assimilation for more flexible and standardized deep learning}},
  year = {2020},
  doi = {10.1101/2020.12.17.423197},
  howpublished = {bioRxiv},
}
Figure