Spontaneous Spatial Cognition Emerges during Egocentric Video Viewing through Non-invasive BCI

Weichen Dai, Yuxuan Huang, Li Zhu, Dongjun Liu, Yu Zhang, Qibin Zhao, Andrzej Cichocki, Fabio Babiloni, Ke Li, Jianyu Qiu, Gangyong Jia, Wanzeng Kong, Qing WuarXiv 2025 · DOI · arXiv

Introduction

Understanding how the human brain represents large-scale spatial information is a fundamental challenge in neuroscience. While invasive studies in animals have identified specialized neurons like place and grid cells, non-invasive observation of these processes in humans remains difficult. Traditional methods like fMRI lack the temporal resolution to capture fast-paced navigation, while EEG often relies on overly simplified stimuli.

Conceptual overview of the 6D spatial information decoding from EEG signals during passive video viewing.

This research demonstrates that non-invasive EEG-based brain-computer interfaces (BCIs) can decode a person's fine-grained 6D pose (3D position and orientation) simply by having them watch first-person (egocentric) video. The study reveals that the brain spontaneously constructs internal spatial maps during passive viewing, provided the visual input maintains chronological continuity.

Methodology

The researchers developed a data-driven framework to bridge the gap between noisy scalp-level EEG signals and precise spatial coordinates. The system employs a coupled training approach using two encoders: one for EEG signals and a ResNet-based model with an Attention module for video frames.

The training process utilized a combination of three loss functions:

  1. Task Loss: A standard regression loss for predicting position and orientation.
  2. Coupled Loss (TripletMarginLoss): This aligns EEG features with visual features, essentially teaching the BCI to interpret neural signals within the spatial context of the video.
  3. Private Loss: This ensures the model captures unique information specific to each modality by enforcing orthogonality between their feature spaces.

During inference, the visual encoder is discarded, and the BCI predicts the 6D pose using only the EEG signals.

The Importance of Temporal Continuity

A critical finding of this study is the "Continuity Requirement." Decoding was only successful when the video was played in its original, chronological order.

Comparison of decoded trajectories between sequential playback and various random/shuffled frame conditions.

As shown in the figure above, the "sequence" condition allows for a coherent reconstruction of the spatial path, whereas "random" conditions result in chaotic, high-error predictions. This suggests that the brain does not merely recognize landmarks but actively integrates motion and time to build a spatial map. Furthermore, decoding accuracy peaked at a presentation rate of 100ms per frame, aligning with the P1 wave in EEG, which suggests a natural "sampling rate" for human spatial cognition.

Key Results and Insights

The BCI achieved remarkably low error rates when decoding from sequential video compared to shuffled frames:

  • Translational Error: Reduced from ~0.88m (random) to 0.08m (sequential).
  • Rotational Error: Reduced from ~40° (random) to 4.8° (sequential).

Attribution mapping revealed a functional dissociation in the brain: position-related signals were concentrated near midline sensorimotor and parietal regions, while orientation relied more on lateral electrodes. This suggests that the brain processes "where I am" and "which way I am facing" through distinct neural pathways.

Conclusion and Future Work

This research marks a significant step toward "brain-inspired navigation." It proves that spatial cognition occurs spontaneously even during passive observation. While the study used 2D video, future work aims to incorporate Virtual Reality (VR) to provide more immersive depth cues and integrate eye-tracking to account for gaze dynamics. These advancements could lead to BCIs that assist paralyzed individuals in navigating wheelchairs or help develop more human-like spatial AI for robotics.

Summary updated

Abstract

Humans possess a remarkable capacity for spatial cognition, allowing for self-localization even in novel or unfamiliar environments. While hippocampal neurons encoding position and orientation are well documented, the large-scale neural dynamics supporting spatial representation, particularly during naturalistic, passive experience, remain poorly understood. Here, we demonstrate for the first time that non-invasive brain-computer interfaces (BCIs) based on electroencephalography (EEG) can decode spontaneous, fine-grained egocentric 6D pose, comprising three-dimensional position and orientation, during passive viewing of egocentric video. Despite EEG's limited spatial resolution and high signal noise, we find that spatially coherent visual input (i.e., continuous and structured motion) reliably evokes decodable spatial representations, aligning with participants' subjective sense of spatial engagement. Decoding performance further improves when visual input is presented at a frame rate of 100 ms per image, suggesting alignment with intrinsic neural temporal dynamics. Using gradient-based backpropagation through a neural decoding model, we identify distinct EEG channels contributing to position -- and orientation specific -- components, revealing a distributed yet complementary neural encoding scheme. These findings indicate that the brain's spatial systems operate spontaneously and continuously, even under passive conditions, challenging traditional distinctions between active and passive spatial cognition. Our results offer a non-invasive window into the automatic construction of egocentric spatial maps and advance our understanding of how the human mind transforms everyday sensory experience into structured internal representations.
BibTeX
@misc{dai2025spontaneous,
  author = {Weichen Dai and Yuxuan Huang and Li Zhu and Dongjun Liu and Yu Zhang and Qibin Zhao and Andrzej Cichocki and Fabio Babiloni and Ke Li and Jianyu Qiu and Gangyong Jia and Wanzeng Kong and Qing Wu},
  title = {{Spontaneous Spatial Cognition Emerges during Egocentric Video Viewing through Non-invasive BCI}},
  year = {2025},
  eprint = {2507.12417},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2507.12417},
}
Figure