Humans flexibly integrate social information despite interindividual differences in reward

Alexandra Witt, Wataru Toyokawa, Kevin N. Laland, Wolfgang Gaissmaier, Charley M. Wu

TL;DR

Researchers have developed a novel "Social Generalization" (SG) model showing that humans don't just blindly imitate others; instead, we treat social information as a useful but noisy signal to guide our own exploration. By testing participants in a "socially correlated bandit task," the study reveals that social learning allows us to integrate information from people with different preferences and can even serve as a "resource-rational" substitute for costly individual exploration.

What Problem Does This Paper Address?

Most research into social learning—how we learn by observing others—operates under a simplifying assumption: that the "demonstrator" and the "observer" share identical goals and reward functions. In the lab, if a demonstrator finds a high-value reward, the observer is usually incentivized to copy them exactly.

In the real world, this is rarely the case. If you're looking for a restaurant in a new city, a 5-star review from a stranger is a useful signal, but their tastes might not perfectly align with yours. This paper addresses the gap between "exact imitation" (copying someone exactly like you) and "asocial learning" (ignoring everyone else). The authors ask: How do humans flexibly use social information when others have similar, but not identical, preferences?

Methods

The authors introduced a socially correlated bandit task. Participants (acting as "scientists" collecting salt samples on alien oceans) searched a 121-tile grid for rewards. Crucially:

  1. Spatial Correlation: High rewards were clustered together (modeled via Gaussian Processes).
  2. Social Correlation: In group rounds, four participants searched their own versions of the grid. These grids were positively correlated ($r \approx 0.6$), meaning a "hotspot" for one person was likely—but not guaranteed—to be a hotspot for another.

The researchers compared four computational models:

  • Asocial Learner (AS): Uses a Gaussian Process Upper Confidence Bound (GP-UCB) to explore based only on personal experience.
  • Decision Biasing (DB): A simple imitation model that increases the probability of choosing what others chose.
  • Value Shaping (VS): Treats others' choices as "pseudo-rewards" that boost the perceived value of an option.
  • Social Generalization (SG): The authors' novel model. It incorporates social observations directly into the Gaussian Process generalization process but treats them as having a specific "social noise" ($\epsilon_{soc}$) component, making them less reliable than personal observations.

The study utilized evolutionary simulations to find the optimal normative strategy and two online experiments (via Prolific) to see which model best described actual human behavior.

Datasets

The study relied on original behavioral data collected from three online experimental groups:

  • Experiment 1 ($N=128$): Group rounds with moderate social correlation ($r=0.6$).
  • Experiment 1R ($N=156$): A control group with very low social correlation ($r=0.1$) to see if humans are sensitive to the relevance of social info.
  • Experiment 2 ($N=132$): A preregistered replication that interleaved "solo" and "group" rounds to establish an asocial baseline for each participant.

Key Insights and Ideas

  • Social Learning as Generalization: Humans don't just copy; we generalize. Participants were more likely to explore near where others found success, rather than just clicking the exact same tile. The SG model captured this "local enhancement" better than traditional imitation models.
  • The "Grain of Salt" Strategy: The best-fitting model (SG) treats social information as inherently noisier than individual experience. This allows humans to benefit from others' successes without being led astray by differences in individual "taste" or reward functions.
  • Resource-Rational Trade-offs: One of the most striking findings was that social learning partially replaces individual exploration. When social info was available, participants' "directed exploration" parameter ($\beta$) decreased. Essentially, observing others allows us to offload the cognitive and opportunity costs of exploring a new environment ourselves.
  • Adaptive Flexibility: Humans are not "obligate" social learners. In Experiment 1R, when the social correlation was dropped to $r=0.1$ (making others' info nearly useless), participants correctly pivoted to an asocial strategy.

Key Results

  • Model Fit: The Social Generalization (SG) model was the clear winner, with a protected exceedance probability ($pxp$) of $\approx 0.98$ in Experiment 1.
  • Performance: Participants who relied more on social information (lower social noise $\epsilon_{soc}$) generally achieved higher mean rewards, provided they didn't use it indiscriminately.
  • Exploration Reduction: In Experiment 2, the directed exploration parameter ($\beta$) was significantly lower in group rounds compared to solo rounds ($p = .004$), supporting the idea that social info acts as an exploration tool.
  • Search Distance: Participants' "social search distance" (how far they clicked from a teammate's previous choice) was significantly lower than random chance but higher than their individual search distance, reflecting a nuanced integration of social data.

Limitations and Future Work

  • Domain Specificity: The task was purely spatial (a 2D grid). The authors acknowledge that while Gaussian Processes work well here, future work should investigate if SG applies to more abstract, non-spatial feature spaces (e.g., choosing a career or a complex product).
  • Group Dynamics: The study focused on how individuals integrate info, but not how groups coordinate. It didn't account for active teaching or the potential for "information cascades" where a group might converge on a sub-optimal solution.
  • Fixed Correlations: The social correlation was fixed within experiments. In reality, we might have "best friends" with $r=0.9$ and "strangers" with $r=0.2$. Future research could explore how humans handle varying levels of correlation within a single group.

Summary updated

Abstract

There has been much progress in understanding human social learning, including recent studies integrating social information into the reinforcement learning framework.Yet previous studies often assume identical payoffs between observer and demonstrator, overlooking the diversity of real-world interactions. We address this gap by introducing a socially correlated bandit task that accommodates payoff differences among participants, allowing for the study of social learning under more realistic conditions. Our novel Social Generalization (SG) model, tested through evolutionary simulations and two online experiments, outperforms existing models by incorporating social information into the generalization process, but treated as noisier than individual observations. Our findings suggest that human social learning is more flexible than previously believed, with the SG model indicating a potential resource-rational trade-off where social learning partially replaces individual exploration. This research highlights the flexibility of humans social learning, allowing us to integrate social information from others with different preferences, skills, or goals.
BibTeX
@inproceedings{witt2024humans,
  author = {Alexandra Witt and Wataru Toyokawa and Kevin N. Laland and Wolfgang Gaissmaier and Charley M. Wu},
  title = {{Humans flexibly integrate social information despite interindividual differences in reward}},
  booktitle = {Proceedings of the National Academy of Sciences},
  year = {2024},
  doi = {10.1073/pnas.2404928121},
}
Figure