Serial grouping of 2D-image regions with object-based attention in humans

Danique Jeurissen, Matthew W. Self, Pieter R. RoelfsemaeLife 2016 · DOI

TL;DR

This paper investigates how our brains group visual elements into coherent objects, proposing a "growth-cone" model where attention spreads serially and its speed depends on the size of image regions. This work sheds light on the dynamic, scale-dependent nature of visual perception and offers a computational framework for understanding object-based attention.

What Problem Does This Paper Address?

When we look at a scene, our visual system doesn't just process individual pixels; it actively groups features into meaningful objects. While we intuitively understand this process as effortless, the underlying mechanisms are complex and not fully understood. Previous theories suggested grouping might be instantaneous if elements are connected. However, this paper questions that assumption, proposing that grouping is a gradual, serial process influenced by how attention spreads across an object. Specifically, they investigate how the time course of this grouping unfolds and how factors like image scale affect its speed, aiming to bridge the gap between low-level feature processing and high-level object perception.

Methods

The researchers employed a psychophysical approach, measuring human participants' reaction times (RTs) in various visual grouping tasks. They designed experiments where participants had to judge if two presented cues belonged to the same object or different objects. To analyze the data, they developed and tested several computational models predicting RTs:

  • Euclidean Model: Assumes RT is purely based on the straight-line distance between cues.
  • Filling-in Model: Predicts RT based on the distance of cues to the object's boundary.
  • Pixel-by-pixel Model: Models grouping as attention spreading serially at a constant speed along the shortest path within the object.
  • Growth-cone Model: The novel model proposed here. It posits that grouping speed is scale-dependent, with attention spreading faster over large, homogeneous regions and slower over narrow or detailed areas. This is conceptualized as a "growth cone" of attention expanding, with its size (and thus speed) dictated by the local scale of the image region.

They used regression analysis and bootstrapping to compare how well each model explained the observed RT data. Experiment 3 specifically used a variation of the Egly cueing paradigm to directly measure the spread of object-based attention.

Datasets

The study used custom-generated stimuli for their experiments. These included:

  • Wedge-shaped objects (Experiment 1).
  • Various image types for object recognition tasks: color pictures, detailed cartoons, cartoon outlines, and scrambled cartoons (Experiment 2).
  • Broad and narrow bars (Experiment 3).

No pre-existing public datasets were used for training or evaluation.

Key Insights and Ideas

  • Grouping is Serial, Not Instantaneous: Contrary to some earlier theories, the results strongly suggest that perceptual grouping is a gradual, step-by-step process, not an instantaneous one. This implies a dynamic mechanism rather than a simple parallel feature binding.
  • Attention Spreads Like a "Growth Cone": The "growth-cone" model, where attention expands incrementally, best explains the observed reaction times. This model captures the idea that grouping isn't uniform but progresses over time.
  • Scale Matters: The speed of this attentional spread is not constant. It's significantly faster in large, uniform areas and slows down considerably in narrow or complex regions. This scale-dependency is a crucial finding.
  • Cortical Hierarchy as an Implementation: The authors propose that the visual cortex's hierarchical structure, with neurons having different receptive field sizes at different levels, could implement this scale-dependent grouping. Larger receptive fields in higher areas would facilitate faster spreading over large regions, while smaller receptive fields in lower areas would be needed for detailed or narrow areas, slowing down the process.
  • Object-Based Attention is Key: The experiments directly demonstrate that grouping relies on object-based attention, which spreads across the entire object, not just local feature connections.

Key Results

  • Growth-Cone Model Dominance: The growth-cone model consistently outperformed other models (Euclidean, filling-in, pixel-by-pixel) in explaining reaction time data across different stimulus types, accounting for up to 76% of the variance in some experiments.
  • Speed Estimates: The estimated speed of attention spreading was around 22-31 ms per "growth cone," suggesting a serial process that takes tens to hundreds of milliseconds for complex scenes.
  • Scale-Dependent Slowdown: Reaction times increased significantly when grouping involved narrow regions or when cues were further from the fixation point, consistent with the growth-cone model's predictions.
  • Object-Based Advantage: Experiment 3 confirmed an object-based advantage in attention spreading, with attention spreading faster for broader objects than narrower ones.

Limitations and Future Work

  • Interior Features and Recognition: The growth-cone model, as implemented, primarily uses object outlines. The authors acknowledge that for detailed images with interior contours, color, and texture, the model's explanatory power decreased. Future work should incorporate mechanisms for object recognition and processing of interior features to better account for grouping in natural scenes.
  • Semantic Segmentation: They suggest that advances in deep learning for semantic segmentation could inform future models, potentially by integrating top-down influences from higher visual areas to lower ones, mimicking how object recognition might guide parsing.
  • Natural Scenes: While the study used simplified 2D shapes and line drawings, the ultimate goal is to understand grouping in complex, naturalistic images. Future research should bridge this gap.
  • Neural Implementation: While they propose a link to the visual cortical hierarchy, further neurophysiological and neuroimaging studies are needed to directly confirm how this scale-dependent spread is implemented in the brain.

Summary updated

Abstract

After an initial stage of local analysis within the retina and early visual pathways, the human visual system creates a structured representation of the visual scene by co-selecting image elements that are part of behaviorally relevant objects. The mechanisms underlying this perceptual organization process are only partially understood. We here investigate the time-course of perceptual grouping of two-dimensional image-regions by measuring the reaction times of human participants and report that it is associated with the gradual spread of object-based attention. Attention spreads fastest over large and homogeneous areas and is slowed down at locations that require small-scale processing. We find that the time-course of the object-based selection process is well explained by a 'growth-cone' model, which selects surface elements in an incremental, scale-dependent manner. We discuss how the visual cortical hierarchy can implement this scale-dependent spread of object-based attention, leveraging the different receptive field sizes in distinct cortical areas.
BibTeX
@inproceedings{jeurissen2016serial,
  author = {Danique Jeurissen and Matthew W. Self and Pieter R. Roelfsema},
  title = {{Serial grouping of 2D-image regions with object-based attention in humans}},
  booktitle = {eLife},
  year = {2016},
  doi = {10.7554/elife.14320},
}
Figure