No single, stable 3D representation can explain pointing biases in a spatial updating task

Jenny Vuong, Andrew Fitzgibbon, Andrew Glennerster

Abstract

People are able to keep track of objects as they navigate through space, even when objects are out of sight. This requires some kind of representation of the scene and of the observer's location but the form this might take is debated. We tested the accuracy and reliability of observers' estimates of the visual direction of previously-viewed targets. Participants viewed four objects from one location, with binocular vision and small head movements then, without any further sight of the targets, they walked to another location and pointed towards them. All conditions were tested in an immersive virtual environment and some were also carried out in a real scene. Participants made large, consistent pointing errors that are poorly explained by any stable 3D representation. Any explanation based on a 3D representation would have to posit a different layout of the remembered scene depending on the orientation of the obscuring wall at the moment the participant points. Our data show that the mechanisms for updating visual direction of unseen targets are not based on a stable 3D model of the scene, even a distorted one.

Introduction

When navigating an environment, humans maintain an awareness of the locations of objects even after they are obscured. Neuroscientists have long debated the nature of this "spatial updating." One prominent theory suggests the brain constructs a stable 3D "cognitive map"—a coordinate-based model that remains consistent as we move. An alternative view is that we rely on "view-based" snapshots linked by movement. This research tests the cognitive map hypothesis: if the brain uses a consistent 3D representation, pointing errors should be explainable by a single, albeit potentially distorted, geometry. If errors shift based on factors irrelevant to a static map—such as the angle of a wall the observer stands behind—the stable 3D model theory becomes untenable.

Methods and Experimental Design

The study employed an immersive Virtual Reality (VR) environment, with key experiments replicated in a physical laboratory to ensure ecological validity.

  1. The Task: Participants viewed four colored targets from a designated "Start Zone."
  2. The Movement: After viewing, participants walked behind a partition (a "wall") that blocked the targets from view, moving to various "Pointing Zones."
  3. The Measurement: From these zones, participants used a hand-held pointer to indicate the direction of the hidden boxes.
Experimental layout showing the virtual environment (b) and the top-down schematic of targets and pointing zones (c).

The researchers manipulated variables such as path length, facing direction, and, most critically, the orientation of the obscuring wall. By changing the wall's slant after the targets were viewed, the team could isolate whether the immediate environment biased the recall of the 3D scene.

Participants indicating the direction of hidden targets using a hand-held pointer. The red and blue lines illustrate the angular measurements used to calculate pointing bias.

Key Insights: The "Wall Effect"

The most significant finding was that pointing errors shifted systematically based on the orientation of the obscuring wall. Because this wall was often not visible or relevant during the initial encoding of the target locations, a stable 3D model cannot explain why it would bias pointing later.

The data suggests that humans are remarkably poor at the "trigonometry" of spatial updating, often making consistent errors of up to 30°. These errors were not caused by "noise" or "drift" during movement, as path length did not affect error magnitude. Instead, the brain appears to use a heuristic: it projects remembered objects onto a 2D plane parallel to the observer's current environment (the wall) rather than maintaining a true 3D volume.

Results and Model Comparison

  • Reliability: A high correlation (0.92) was found between errors in real-world and virtual environments, validating VR as a robust tool for spatial cognition research.
  • Model Performance: A "projection-plane" model—a zero-parameter heuristic—consistently outperformed a 3D model that was granted 24 free parameters to fit the data. This strongly indicates that no single 3D layout can account for human performance.
  • Consistent Biases: In Experiment 4, altering only the wall orientation caused pointing directions to shift in opposite directions, directly contradicting the existence of a stable internal representation.

Conclusion

This study demonstrates that human spatial updating is not based on a stable 3D model of the scene. Instead, the brain relies on task-dependent heuristics that result in predictable, systematic biases. While real-world landmarks likely help "reset" these internal maps and reduce errors in daily life, the underlying mechanism for tracking unseen objects is fundamentally non-geometric. Future research may find that deep reinforcement learning models, which navigate without explicit 3D maps, provide a more accurate reflection of human spatial cognition.

Summary updated

Abstract

People are able to keep track of objects as they navigate through space, even when objects are out of sight. This requires some kind of representation of the scene and of the observer's location but the form this might take is debated. We tested the accuracy and reliability of observers' estimates of the visual direction of previously-viewed targets. Participants viewed four objects from one location, with binocular vision and small head movements then, without any further sight of the targets, they walked to another location and pointed towards them. All conditions were tested in an immersive virtual environment and some were also carried out in a real scene. Participants made large, consistent pointing errors that are poorly explained by any stable 3D representation. Any explanation based on a 3D representation would have to posit a different layout of the remembered scene depending on the orientation of the obscuring wall at the moment the participant points. Our data show that the mechanisms for updating visual direction of unseen targets are not based on a stable 3D model of the scene, even a distorted one.
BibTeX
@inproceedings{vuong2019no,
  author = {Jenny Vuong and Andrew Fitzgibbon and Andrew Glennerster},
  title = {{No single, stable 3D representation can explain pointing biases in a spatial updating task}},
  booktitle = {Scientific},
  year = {2019},
  doi = {10.1038/s41598-019-48379-8},
}
Figure