Structured behavioral data format: An NWB extension standard for task-based behavioral neuroscience experiments

Ryan Ly, Matthew Avaylon, Michael Wulf, Ádám Kepecs, Oliver RuebelbioRxiv 2024 · DOI

TL;DR

Neuroscientists need a standardized way to store complex behavioral data alongside neural recordings. This paper introduces ndx-structured-behavior, an extension to the Neurodata Without Borders (NWB) standard, to capture detailed behavioral task information, from stimulus delivery to subject responses, enabling richer, integrated analyses of brain function.

What Problem Does This Paper Address?

Understanding how the brain generates behavior requires simultaneously observing neural activity and the intricate details of an organism's actions. However, current methods for recording and storing behavioral data from neuroscience experiments are often ad-hoc, relying on custom-built systems and diverse data formats. This lack of standardization makes it difficult to share, reproduce, and integrate behavioral data with neural recordings, hindering progress in understanding complex neural computations. Specifically, there's a need for a comprehensive data standard that can fully encapsulate task-based experiments, linking the richness of behavioral context with neural activity.

Methods

The core contribution of this paper is the development of ndx-structured-behavior, a formal extension to the existing NWB data standard. NWB is a community-driven standard for neurophysiology data, and this extension builds upon it to specifically address the needs of behavioral neuroscience.

The ndx-structured-behavior extension introduces several key components:

  • Task Description: Metadata about the behavioral task itself is stored in a /general group. This includes:
  • TaskProgram: A text dataset storing the raw task program definition (e.g., in JSON, XML, or Python scripts).
  • TaskSchema: An optional schema to help parse and understand the TaskProgram.
  • TaskArgumentsTable: Stores parameters of the task program, their descriptions, and how their values are determined (e.g., constants or functions).
  • StatesTypesTable, EventsTypesTable, ActionsTypesTable: These tables define the possible states, events, and actions within a task, providing a controlled vocabulary.
  • Task Recording: Actual experimental data is stored in the /acquisition group. This includes:
  • StatesTable, EventsTable, ActionsTable: These tables record the timing and occurrence of states, events, and actions during the experiment, referencing the types defined above.
  • TrialsTable: This table annotates the temporal structure of the experiment by defining trials and linking them to the recorded states, events, and actions. This allows for easy navigation and analysis of trial-specific data.

The extension leverages NWB's DynamicTable and TimeIntervals types for flexible data storage and querying. The authors also developed tools to convert data from popular behavioral control systems like BAABL and ArControl into this NWB format.

Datasets

The paper does not explicitly mention training datasets. However, it uses data from a "light chasing task" implemented using the BAABL control system as a primary example to demonstrate and validate the ndx-structured-behavior extension. This task involves a subject in a behavior box with three ports, where a correct port is randomly chosen and rewarded. The data from this task is used to generate visualizations and analyses, showcasing the utility of the new standard.

Key Insights and Ideas

  • Bridging the Gap: The most significant insight is the creation of a standardized bridge between complex behavioral task definitions and the actual recorded data, including neural activity. This addresses a critical bottleneck in neuroscience research.
  • Leveraging Existing Standards: Building upon the established NWB standard is a smart move, allowing researchers to integrate behavioral data into existing neurophysiology workflows without starting from scratch. This promotes adoption and interoperability.
  • Structured Representation is Key: The paper highlights that simply storing raw logs isn't enough. A structured representation of states, events, and actions, linked to the task's logic, is crucial for meaningful analysis and comparison across experiments.
  • Enabling FAIR Data: By providing a standardized format, this extension directly contributes to making behavioral neuroscience data more Findable, Accessible, Interoperable, and Reusable (FAIR).

Key Results

The paper demonstrates the successful application of the ndx-structured-behavior extension by:

  • Storing Task Metadata: Successfully capturing the detailed description of the light chasing task, including its parameters, states, events, and actions.
  • Recording Behavioral Data: Storing the timestamped occurrences of states, events, and actions during the experiment.
  • Defining Trial Structure: Organizing the recorded data into meaningful trials, linking them to specific behavioral sequences.
  • Enabling Visualization: Generating time raster plots of events, actions, and states, as well as state transition graphs, which are essential for visualizing and understanding behavioral dynamics.
  • Facilitating Analysis: The structured data allows for quantitative analysis, such as calculating state transition probabilities, as demonstrated in the light chasing task example.

Limitations and Future Work

The authors acknowledge several limitations and suggest future directions:

  • Task Program Interoperability: A significant challenge remains in harmonizing and enabling interoperability of task programs across different control software and hardware. The current approach of storing raw task programs (e.g., in BAABL's XML) is hardware-specific and hinders reuse. More effort is needed to create hardware-agnostic, portable task description languages.
  • Tooling for Task Evaluation: The community currently lacks tools to formally evaluate behavioral tasks before running experiments in the field. This can lead to late detection of errors and hinders computational modeling.
  • Community Adoption: While the extension is presented, its widespread adoption will depend on community engagement and the development of supporting tools and workflows.

Future work could focus on developing more sophisticated tools for task program authoring and validation, as well as expanding the ecosystem of converters for other behavioral control systems.

Summary updated

Abstract

Abstract Understanding brain function necessitates linking neural activity with corresponding behavior. Structured behavioral experiments are crucial for probing the neural computations and dynamics underlying behavior; however, adequately representing their complex data is a significant challenge. Currently, a comprehensive data standard that fully encapsulates task-based experiments, integrating neural activity with the richness of behavioral context, is lacking. We designed a data model, as an extension to the NWB neurophysiology data standard, to represent structured behavioral neuroscience experiments, spanning stimulus delivery, timestamped events and responses, and simultaneous neural recordings. This data format is validated through its application to a variety of experimental designs, showcasing its potential to advance integrative analyses of neural circuits and complex behaviors. This work introduces a comprehensive data standard designed to capture and store a spectrum of behavioral data, encapsulating the multifaceted nature of modern neuroscience experiments.
BibTeX
@misc{ly2024structured,
  author = {Ryan Ly and Matthew Avaylon and Michael Wulf and Ádám Kepecs and Oliver Ruebel},
  title = {{Structured behavioral data format: An NWB extension standard for task-based behavioral neuroscience experiments}},
  year = {2024},
  doi = {10.1101/2024.01.08.574597},
  howpublished = {bioRxiv},
}
Figure