Collective Gaze Aggregation for Shared Attention

You are reading version 1 of this direction, frozen 2026-10-04. The latest version is version 2.

Version 1 of 215 papers22 conceptsfrozen 2026-10-04

When we look at a scene, our eyes rarely stay still. We scan, we wander, and we occasionally fixate on irrelevant details. For a computer vision system, a single person's gaze is a noisy signal—a mix of intentional focus and idiosyncratic visual "noise." However, when multiple people view the same scene, their individual wandering cancels out, leaving behind a clear, robust signal of what truly matters.

This research direction focuses on collective gaze aggregation: the process of combining eye-tracking data from multiple observers to identify shared attention, extract salient objects, and understand social dynamics.

The Signal in the Noise

Individual gaze is notoriously difficult to use as a ground truth for "importance." One person might be distracted by a lens flare, while another might be looking at a person's shoes instead of their face. S. Karthikeyan 0001, Thuyen Ngo, and colleagues demonstrate in Eye tracking assisted extraction of attentionally important objects from videos that aggregating these patterns acts as a "weak supervisor." By overlaying the gaze of multiple viewers, the system filters out individual distractions to highlight universally salient elements, such as moving vehicles or faces, even when traditional motion-based algorithms fail.

This concept is critical in high-stakes environments like autonomous driving. Ye Xia and David Whitney found that a "multi-focus attention map"—created by aggregating where many drivers look—captures potential hazards more reliably than any single driver could. This collective map identifies "covert attention," spotting hazards that a single-point tracker might miss because the driver was momentarily checking their mirrors.

Spatial Convergence and Heatmaps

The most intuitive way to aggregate gaze is to look for where sightlines intersect in 3D space or overlap on a 2D plane. Lifeng Fan and Wenguan Wang formalize this in Inferring Shared Attention in Social Scene Videos by projecting individual gaze vectors into a unified spatial heatmap. Peaks in this heatmap represent "shared attention"—the specific objects or regions that have successfully captured the group's collective interest.

To quantify how "focused" a group is, researchers like Arnab Sen Sharma use metrics like gaze cohesiveness. By calculating the mean squared error of individual gaze points relative to their collective centroid, Sharma et al. can distinguish between a group that is collectively engaged with a specific lecture slide and a group whose attention has fractured.

Temporal Synchronization and Social Dynamics

Collective gaze isn't just about where people look, but when they look at it together. In collaborative settings, the degree of synchronization between partners can reveal the "health" of the interaction. Ulrike Creß and Gerry Stahl use cross-recurrence analysis to measure this synchronization, identifying whether a pair is working as a balanced partnership or a leader-follower dyad.

This temporal alignment has powerful applications in education. Songlin Xu and Xinyu Zhang showed that by visualizing "peer attention regions"—showing new students where previous successful learners focused—they could synchronize the gaze of a distributed group. This collective alignment reduced confusion and significantly boosted test scores for complex concepts.

Automating Social Intelligence

By understanding where a group's attention is focused, machines can make smarter decisions about social interactions. Hui-Yin Wu developed a system for automated video editing that uses a "global focus score." Instead of just following the person speaking, the system tracks the collective head poses of everyone in the room. If the entire group suddenly looks at a whiteboard, the camera cuts to the whiteboard, capturing the essential group dynamics that an audio-only system would miss.

Similarly, in the realm of conversational AI, Seongsil Heo and Calvin Murdock use multi-user attention mechanisms to predict turn-taking. By integrating the gaze of all participants, their model can sense when a secondary speaker is about to jump in, allowing for more natural interactions in augmented reality and social robotics.

Beyond Interaction: Clinical Biomarkers

Finally, the patterns of how we aggregate our attention can serve as objective biological markers. Frederick Shic and Adam Naples demonstrate that aggregating gaze data across many different social stimuli—like videos of children playing or static faces—creates a robust "biomarker" for autism. By looking at the collective response across various contexts, researchers can reduce the noise of individual trials and create a stable measure of social attention for use in clinical trials.

Papers in this version

History & citationVersion 2 · 2026-10-04

Current · Permanent link

  1. Version 2 (latest) · 2026-10-04 · 19 papers
  2. Version 1 · 2026-10-04 · 15 papers
Cite this direction

Latest version (a living page):

@misc{lacuna_multi_observer_gaze_aggregation_36a938e3,
  title = {multi-observer gaze aggregation},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/multi-observer-gaze-aggregation/rdr_36a938e3875b40069baae7286c05b45a},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_collective_gaze_aggregation_for_shared_a_36a938e3_v1,
  title = {Collective Gaze Aggregation for Shared Attention},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/collective-gaze-aggregation-for-shared-attention/rdr_36a938e3875b40069baae7286c05b45a/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}