Abstract
Most mental processes consist of a number of processing steps that are executed sequentially. The timing of the individual mental operations can usually only be estimated indirectly, from the pattern of reaction times. In vision, however, many processing steps are associated with the modulation of neuronal activity in early visual areas. Here we exploited this association to elucidate the time course of neuronal activity related to each of the self-paced mental processing steps in complex visual tasks. We trained monkeys to perform two tasks, search-trace and trace-search, which required performing a sequence of two operations: a visual search for a specific color and the mental tracing of a curve. We used multielectrode recording techniques to monitor the representations of multiple visual items in area V1 at the same time and found that the relevant curve as well as the target of visual search evoked enhanced neuronal activity with a timing that depended on the order of operations. This modulation of neuronal activity in early visual areas could allow these areas to (1) act as a cognitive blackboard that permits the exchange of information between successive processing steps of a sequential visual task and to (2) contribute to the orderly progression of task-dependent endogenous attention shifts that are driven by task structure and evolve over hundreds of milliseconds.
Introduction: Beyond Passive Filtering
Most complex mental processes are composed of discrete "atomic" operations executed in sequence. Traditionally, the timing of these internal steps has been inferred indirectly from aggregate reaction times. However, this study demonstrates that the primary visual cortex (V1)—long considered a low-level sensory area—actively participates in the scheduling and execution of these sequential routines. By recording directly from V1 in macaques, researchers have shown that early visual areas act as a "cognitive blackboard," revealing the precise temporal order of mental sub-steps like color searching and curve tracing.
Experimental Design and Visual Routines
To investigate the neural signatures of sequential processing, the researchers trained two macaque monkeys (Macaca mulatta) to perform two distinct visual routines using identical components but in reversed order:
- Trace-then-Search: The subject traces a curve from a starting point to identify a colored marker, then searches for a larger disc of that same color.
- Search-then-Trace: The subject searches for a marker matching the fixation point's color, then traces the attached curve to a target disc.
The team employed multielectrode recording at 77 distinct sites in V1 to monitor neuronal response modulation—a boost in firing rate that occurs when a visual item becomes task-relevant. By analyzing the timing of these activity boosts, they could map the transition between mental sub-operations.
V1 as a Cognitive Blackboard
The study's central finding is that V1 functions as a workspace where information is "written" and "read" by different brain regions. This "cognitive blackboard" architecture provides a biological solution to the "parameter transfer" problem: how the brain passes the result of one subroutine (e.g., the location of a found marker) to the next (e.g., the start of a tracing operation). V1 provides a retinotopic coordinate system where one process can "mark" a location for the next to utilize.
The timing of these neuronal boosts aligned perfectly with the task structure. In the Trace-then-Search condition, neurons representing the curve showed enhanced activity first, followed by those representing the search target. When the task order was reversed, the neural sequence followed suit, proving that V1 activity reflects the logical flow of the mental program.
Quantitative Results
The researchers identified several key temporal milestones in these cognitive operations:
- Initial Evolution: The first mental operation typically takes approximately 200 ms to manifest as a modulation in V1.
- Sequential Hand-off: In the Trace-then-Search task, tracing modulation began at ~180 ms, with search modulation following at ~267 ms. In the reversed Search-then-Trace task, search modulation occurred at ~228 ms, followed by tracing at ~287 ms.
- Complexity Costs: Introducing a second operation added a delay of 60 to 240 ms, depending on the task and the spatial distance between items. This suggests that endogenous attention shifts incur a physical processing cost associated with distance.
Discussion and Future Directions
While V1 displays the execution of these sequences, the "executive program" likely originates in higher-order regions such as the frontal eye fields (FEF) or the prefrontal cortex. Future research is needed to determine how these areas coordinate the hand-off between V1 subroutines. Additionally, while this study establishes a strong correlation, causal techniques like optogenetics or TMS will be necessary to confirm if V1's "blackboard" activity is strictly required for task completion. Finally, extending these findings from simple geometric stimuli to complex, naturalistic scenes remains a critical next step in understanding real-world visual cognition.
Summary updated