Sculpting New Visual Categories into the Human Brain

Coraline Rinn Iordan, Victoria J. H. Ritvo, Kenneth A. Norman, Nicholas B. Turk‐Browne, Jonathan D. CohenbioRxiv 2020 · DOI

TL;DR

This study introduces a novel method called "neural sculpting" using real-time fMRI neurofeedback to directly manipulate brain activity patterns, creating new visual categories in participants' brains without their explicit awareness. This technique successfully altered both neural representations and perceptual behavior, demonstrating a causal link between sculpted brain activity and category perception, and opening new avenues for understanding and accelerating learning.

What Problem Does This Paper Address?

Traditionally, learning new categories involves acquiring knowledge through experience, study, or instruction. While neuroimaging has revealed neural correlates of learning, it has primarily been correlational. This paper tackles the challenge of establishing a causal link between specific neural representations and perceptual learning. Specifically, it investigates whether directly sculpting neural activity patterns in the brain, mimicking those expected from learning, can induce new perceptual categories and alter behavior, even in the absence of explicit instruction or prior experience with those categories. This moves beyond simply reinforcing existing neural patterns to actively creating new ones.

Methods

The core of this research is closed-loop real-time functional magnetic resonance imaging (fMRI) neurofeedback. Participants were presented with complex visual shapes generated from radial frequency components. The experiment involved:

  • Stimulus Construction: A two-dimensional manifold of shapes was created by varying the amplitudes of two radial frequency components, allowing for smooth transitions between shapes. Arbitrary category boundaries were defined along specific diameters of this shape space.
  • Behavioral Norming: A psychophysical experiment was conducted to ensure participants had no pre-existing biases in categorizing shapes along these defined diameters.
  • fMRI Localizer Scans: These scans were used to map individual participants' brain representations of the shape space and identify regions of interest (ROIs) for neurofeedback. A searchlight analysis was employed to find brain regions that parametrically represented the shape space.
  • Neural Sculpting (Neurofeedback Training): Participants underwent several days of real-time fMRI training. They were instructed to mentally influence a continuously oscillating shape on screen to make it "wobble less or stop." Unbeknownst to them, positive feedback (reduced oscillation) was provided when their brain activity in the target ROI matched a desired pattern associated with a sculpted category. This process aimed to make neural representations of shapes within a category more similar to each other and distinct from shapes in other categories.
  • Neural Modeling: A multivariate Gaussian model was built to represent the neural patterns associated with the two categories defined by the sculpted boundary. This model was used for real-time decoding and feedback.
  • Behavioral Post-tests: After training, participants repeated the 2AFC task to assess changes in perceptual categorization.
  • Statistical Analysis: Various statistical methods, including ANOVAs, t-tests, correlation analyses, and bootstrap resampling, were used to analyze neural and behavioral data and assess the robustness of the findings.

Datasets

  • Stimulus Set: A novel set of complex visual shapes was generated for this study, forming a two-dimensional manifold. This set served as the basis for both behavioral and fMRI experiments.
  • Participant Data: Data was collected from 10 participants who completed the full experimental protocol. This included behavioral data from pre- and post-tests, and fMRI data from localizer and neurofeedback training sessions.

Key Insights and Ideas

  • Direct Neural Manipulation for Category Learning: The study demonstrates that it's possible to directly sculpt neural representations in the brain to create new visual categories, bypassing traditional learning methods. This is a significant step towards understanding the causal role of specific neural patterns in perception.
  • "Sculpting the Neural Clay": The metaphor of "sculpting the neural clay" effectively captures the idea of actively shaping brain activity to form new perceptual structures, echoing ancient philosophical ideas about the creation of concepts.
  • Neural Similarity Drives Categorical Perception: The findings suggest that making neural representations of objects more similar within a category is sufficient to induce stronger categorical perception, even if these categories are arbitrary and not based on pre-existing knowledge.
  • Non-Invasive Causal Link: This work provides a non-invasive method to establish a causal link between specific neural representations and behavior, which is a powerful tool for future neuroscience research.
  • Broader Implications for Cognition: The success in sculpting visual categories suggests potential applications for influencing other cognitive domains like decision-making, memory, and motor control.

Key Results

  • Neural Sculpting Success: Participants showed significant changes in neural representations, with stronger separation between the trained categories compared to control categories. This was measured by an increase in the log-likelihood ratio (LLR) between trained categories.
  • Behavioral Changes: The neural sculpting also led to altered perceptual behavior. Participants exhibited stronger categorical perception for the trained categories, as indicated by a significant increase in the slope of their psychometric functions.
  • Correlation Between Neural and Behavioral Changes: A strong positive correlation was found between the neural changes (LLR) and behavioral changes (psychometric function slope), further supporting the causal link.
  • Individual Variability: While the overall effect was significant, there was variability in the magnitude of the effects across participants, suggesting individual differences in susceptibility to neurofeedback.
  • No Explicit Awareness: Participants were not explicitly informed about the category boundaries or the nature of the study, indicating that the learning occurred implicitly.

Limitations and Future Work

  • Participant Fatigue: The authors acknowledge that participant fatigue during the later stages of the experiment might have influenced behavioral performance.
  • Durability of Effects: The study provides evidence for immediate changes, but the long-term durability of these sculpted neural categories and their behavioral effects remains an open question for future research.
  • Identifying Specific Brain Regions: While a neurofeedback ROI was defined, the specific contribution of individual brain regions within that ROI to the observed effects is not fully elucidated. Future work could focus on pinpointing these critical regions.
  • Generalizability to Complex Concepts: The study used abstract shapes. Future research could explore whether this neural sculpting approach can be applied to more complex, real-world objects or concepts.
  • Clinical Applications: The authors suggest that this method holds promise for clinical applications, such as treating neuropsychiatric disorders or enhancing cognitive functions, but further investigation is needed to explore these possibilities.
  • Strategy Identification: While participants employed varied strategies during neurofeedback, no single strategy was identified as universally effective. Future work could investigate the role of specific strategies in successful neural sculpting.

Summary updated

Abstract

Abstract Learning requires changing the brain. This typically occurs through experience, study, or instruction. We report a proof-of-concept for a new way for humans to acquire visual knowledge by directly sculpting activity patterns in the human brain that mirror those expected to arise through learning. We used a non-invasive technique (closed-loop real-time functional magnetic resonance imaging neurofeedback) to create new categories of visual objects in the brain, without the participants’ explicit awareness. After neural sculpting, participants exhibited behavioral and neural biases for the sculpted, but not for the control categories. The ability to sculpt new perceptual distinctions in the human brain offers a new paradigm for human fMRI research that allows for non-invasive, causal testing of the link between neural representations and behavior. As such, beyond its current application to perception, our work potentially has broad relevance to other domains of cognition such as decision-making, memory, and motor control. Significance Statement Objects that belong to the same category tend to elicit similar patterns of brain activity. Here we reverse this mapping and ask whether neural similarity is sufficient to induce increased perceptual discrimination and categorical perception. We do this by using real-time fMRI to modify neural representations of objects in high-level visual cortex. Participants viewed an object and received closed-loop neurofeedback that pushed them to represent the object more similarly to a brain activity pattern we chose for that category. After successfully self-modulating their brain activity, participants began to perceive objects assigned to the same brain pattern as more categorically distinct from those assigned to a different brain pattern. These findings open a new avenue for understanding and accelerating human learning.
BibTeX
@misc{iordan2020sculpting,
  author = {Coraline Rinn Iordan and Victoria J. H. Ritvo and Kenneth A. Norman and Nicholas B. Turk‐Browne and Jonathan D. Cohen},
  title = {{Sculpting New Visual Categories into the Human Brain}},
  year = {2020},
  doi = {10.1101/2020.10.14.339853},
  howpublished = {bioRxiv},
}
Figure