Encoding of continuous perceptual choices in human early visual cortex

Riccardo Barbieri, Felix M. Töpfer, Joram Soch, Carsten Bogler, Henning Sprekeler, John­–Dylan HaynesFrontiers in Human Neuroscience 2023 · DOI

Perceptual decision-making research has long been dominated by "this or that" scenarios—is the dot moving left or right? Is the image a face or a house? While these binary tasks are easy to model, they don't capture the richness of the real world, where features like color, orientation, and motion exist on a continuous spectrum.

A recent paper published in Frontiers in Human Neuroscience by Barbieri et al. shifts the focus from categorical bins to the full 360-degree continuum. Their findings challenge the traditional view that decisions are primarily "prepared" in higher-order motor or parietal regions, suggesting instead that our early sensory cortex plays a starring role in representing our choices.

TL;DR

Researchers used fMRI and Gaussian process regression to show that the human early visual cortex encodes continuous perceptual choices (e.g., any angle from 0–360°). Crucially, these choice signals were present even when participants were purely guessing, suggesting that sensory areas are actively recruited to represent the outcome of a decision.

What Problem Does This Paper Address?

Most neuroscientific models of decision-making rely on the "intentional framework," which suggests that decisions are encoded as motor plans (e.g., "I will press the left button"). This usually involves the parietal and prefrontal cortices.

However, when a choice is continuous—like identifying the exact shade of a sunset or the precise direction of a gust of wind—we don't always have a pre-defined motor plan. The authors sought to determine where the brain encodes these "feature-continuous" choices. Does the brain still use higher-order regions, or does it "recruit" the same sensory areas that originally processed the stimulus to hold the final decision?

Methods

The researchers tasked 23 participants with viewing Random Dot Kinematograms (RDKs)—clouds of dots moving in a specific direction.

The Continuous Task

Unlike typical studies, the motion could be any angle between 0° and 360°. Participants reported the direction by waiting for a rotating bar to match their perceived direction and then pressing a button. This decouples the choice from the motor action, as the participant doesn't know when they will press the button until the bar rotates into place.

Voxel-wise Gaussian Process Regression (GPR)

To analyze the fMRI data, the team used a sophisticated encoding model based on Gaussian Process Regression.

  • Why GPR? Standard models often assume a specific shape for how a brain cell (or voxel) responds to a stimulus (like a bell curve). GPR is "non-parametric," meaning it can flexibly learn the unique "tuning function" of every single voxel without prior assumptions about its shape.
  • Reconstruction: They used these tuning functions to "read out" or reconstruct the trial-by-trial direction of both the physical stimulus and the participant's reported choice from brain activity.

Datasets

  • Primary Data: fMRI scans and behavioral responses from 23 healthy participants.
  • Conditions: Three levels of motion coherence: 0% (pure noise/guessing), intermediate (~19%), and 100% (perfect signal).
  • Localizers: A separate fMRI task was used to identify Area MT+, a region specifically known for motion processing.

Key Insights and Ideas

  • Sensory Recruitment: The Early Visual Cortex (EVC)—the first stop for visual information in the brain—was the primary site for choice encoding. This suggests that for continuous features, the brain might use sensory areas as a "blackboard" to write down the final decision.
  • The "Guessing" Proof: The most striking finding came from the 0% coherence condition. In these trials, there was no physical motion direction; participants were guessing. Yet, the researchers could still reconstruct the reported direction from the EVC. This proves the signal wasn't just a reflection of the stimulus, but a representation of the participant's internal choice.
  • Parietal Silence: Surprisingly, regions like the parietal cortex, which are usually "hotspots" for categorical decisions, showed very little information about the continuous choices. This suggests that the parietal cortex's role might be specific to tasks where choices are mapped to specific motor actions.

Key Results

  • Reconstruction Accuracy: In the EVC, reconstruction of both stimulus and choice was highly successful and improved with stimulus coherence ($p < 0.001$).
  • Generalization: A model trained on "pure guesses" (0% coherence) could successfully predict the stimulus direction in high-coherence trials ($p = 0.004$). This indicates that the brain uses the same neural "code" for both seeing a real stimulus and making a subjective choice.
  • MT+ Performance: While MT+ is vital for motion, it provided significantly less information for reconstruction than the EVC in this specific continuous task.

Limitations and Future Work

The authors acknowledge a few key hurdles:

  1. Working Memory vs. Decision: Because there is a brief delay between seeing the dots and reporting the direction, it is difficult to distinguish if the EVC is encoding the decision or simply holding the information in working memory.
  2. fMRI Resolution: The lack of signal in MT+ might be a limitation of fMRI's spatial resolution rather than a lack of neural information. Future studies using higher field strengths (e.g., 7T fMRI) might find more nuanced tuning.
  3. Motor Preparation: While the rotating bar decouples specific button presses, the task still ends in a motor act. Future research could explore choices that require no motor output at all to further isolate the "pure" perceptual decision.

Source Paper: Barbieri, R., Töpfer, F. M., et al. (2023). Encoding of continuous perceptual choices in human early visual cortex. Frontiers in Human Neuroscience. DOI: 10.3389/fnhum.2023.1277539

Summary updated

Abstract

We found only little information related to the decision outcome in regions beyond visual cortex, such as parietal cortex, possibly because our task did not involve differential motor preparation. This could suggest that decisions for continuous stimuli take can place already in sensory brain regions, potentially using similar mechanisms to the sensory recruitment in visual working memory.
BibTeX
@inproceedings{barbieri2023encoding,
  author = {Riccardo Barbieri and Felix M. Töpfer and Joram Soch and Carsten Bogler and Henning Sprekeler and John­–Dylan Haynes},
  title = {{Encoding of continuous perceptual choices in human early visual cortex}},
  booktitle = {Frontiers in Human Neuroscience},
  year = {2023},
  doi = {10.3389/fnhum.2023.1277539},
}
Figure