Abstract
While human vision spans 220°, traditional functional MRI setups display images only up to central 10-15°. Thus, it remains unknown how the brain represents a scene perceived across the full visual field. Here, we introduce a method for ultra-wide angle display and probe signatures of immersive scene representation. An unobstructed view of 175° is achieved by bouncing the projected image off angled-mirrors onto a custom-built curved screen. To avoid perceptual distortion, scenes are created with wide field-of-view from custom virtual environments. We find that immersive scene representation drives medial cortex with far-peripheral preferences, but shows minimal modulation in classic scene regions. Further, scene and face-selective regions maintain their content preferences even with extreme far-periphery stimulation, highlighting that not all far-peripheral information is automatically integrated into scene regions computations. This work provides clarifying evidence on content vs. peripheral preferences in scene representation and opens new avenues to research immersive vision.
Introduction
Human vision is naturally expansive, spanning roughly 220° of the horizontal arc. However, due to the physical constraints of MRI scanners—specifically the narrow bore and the use of head coils—most neuroimaging studies of the visual system are limited to a "postcard-sized" view of about 10–20°. This creates a significant gap in our understanding of "immersive" vision. It remains unclear whether the neural mechanisms identified for scene perception (e.g., recognizing a kitchen or a forest) hold true when those scenes wrap around the viewer, or if the far periphery recruits entirely different cortical real estate. This study seeks to determine if "scene-selective" regions are primarily "peripheral-preferring" regions or if they represent a higher-order stage of processing robust to changes in scale and immersion.
Methods
To bypass the field-of-view (FOV) bottleneck, a novel hardware and software solution was implemented.
Hardware and Projection
The top half of the fMRI head coil was removed and replaced with a custom-built, 3D-printed curved screen installed inside the scanner bore. An "inverted periscope" setup using two angled mirrors bounced the projector's image directly onto this screen, achieving an unobstructed 175° view.
Computational Warping and Stimuli
To prevent the "funhouse mirror" effect of projecting a flat image onto a curved surface at an angle, a custom warping algorithm was developed. This ensured the geometry of the scene remained perceptually accurate for the participant.
The study utilized several custom-generated stimulus sets:
- 3D Indoor Scenes: 20 distinct virtual environments rendered in the Unity game engine with a native wide-angle perspective.
- Controlled Arrays: Face and object arrays adjusted for "cortical magnification" in the periphery.
- Retinotopic Stimuli: Flashing checkerboards extended to five levels of eccentricity.
Results
Mapping the Far Periphery
The study successfully mapped the human visual cortex up to >55° of eccentricity (radius), significantly extending standard fMRI maps. A clear functional dissociation was identified: the Peripheral-POS (parieto-occipital sulcus) region is sensitive to spatial location and size (periphery), while classic scene regions like the Parahippocampal Place Area (PPA) and Occipital Place Area (OPA) are driven by image content.
Content vs. Immersion
Classic scene-selective regions (PPA, OPA) are not merely "periphery detectors." While they show a slight boost from full-field images, their primary function is analyzing scene content. They responded most strongly to intact scenes regardless of size or immersion level.
Robustness of Category Selectivity
In an extreme "scotoma" experiment where the central 138° of the visual field was blocked, the Fusiform Face Area (FFA) still showed a preference for faces, and the Retrosplenial Complex (RSC) preferred scenes. This suggests a robust "far-peripheral route" for category-selective processing that does not require central input. Participants could still perform basic categorization at above-chance levels, mirroring this neural activity.
Discussion
The findings suggest a hierarchical organization on the medial surface of the brain. Scene-selective regions are anatomically adjacent to the far-peripheral parts of the early visual cortex, suggesting a "conveyor belt" of information from raw peripheral mapping to high-level representation. The RSC appears to act as a bridge between retinotopic maps and high-level spatial memory, showing stronger sensitivity to the far periphery than the PPA or OPA.
Future work should address current limitations, such as the low pixel density (2–4 pixels per degree) inherent in spreading a 1024x768 image over 175°. Additionally, integrating computational eye-tracking from fMRI images will be crucial, as traditional eye-tracking is difficult in this immersive setup.
Source: Nature Communications (2024) 15:5477 | DOI: 10.1038/s41467-024-49669-0
Summary updated