Diversity and Exploration in Social Learning

Nicole Immorlica, Jieming Mao, Christos TzamosWWW 2019 · OpenReview

In social learning environments, individuals often rely on the experiences of their predecessors to guide their own decisions. Whether choosing a restaurant based on online reviews or selecting a career path based on alumni success, the observation of past choices significantly influences current behavior. However, this reliance can lead to "herding," where a population converges on a sub-optimal choice because early adopters happened to find it "good enough," leaving superior alternatives unexplored.

Traditional economic theory suggests that overcoming this inefficiency requires centralized intervention, such as a platform designer hiding information to force exploration. This paper explores an alternative, decentralized mechanism: preference diversity. By modeling consumer search with varying levels of subjective taste, the authors demonstrate that intermediate levels of diversity naturally incentivize exploration, leading to higher long-term social utility without the need for external coordination.

The Model: Searching for Quality in a Diverse World

The study employs a sequential model of consumer search based on the classic "Pandora’s Box" problem. Agents arrive one by one and face a set of items. Each item $i$ at time $t$ has a value $v_{i,t}$ composed of two parts:

  1. Common Quality ($q_i$): An objective value shared by all agents.
  2. Subjective Score ($s_{i,t}$): A private value unique to agent $t$.

The total variance of the value is fixed, and diversity ($\sigma^2$) is defined as the proportion of this variance attributed to the subjective component. When $\sigma^2 = 0$, agents are identical; when $\sigma^2 = 1$, tastes are entirely private, and no information can be gleaned from others.

Agents use the Weitzman index, an optimal stopping rule, to decide which items to explore. In this "revealed quality" model, once an agent pays a cost to explore an item, its common quality component ($q_i$) is revealed to all subsequent agents. The central question is how the variance $\sigma^2$ affects the average utility of the population over time.

The Goldilocks Principle of Diversity

The research identifies a "Goldilocks" effect where social utility is maximized at intermediate levels of diversity:

  • Low Diversity ($\sigma^2 \to 0$): In a homogeneous population, agents quickly converge on the first "decent" item discovered. Because everyone values items identically, there is no individual incentive to deviate from a known good option to explore unknown ones. This leads to premature herding and low long-term utility.
  • High Diversity ($\sigma^2 \to 1$): When tastes are purely subjective, agents search extensively for their own "best" match. However, because their preferences are unique, the discoveries made by one agent provide no useful information to the next. The social learning mechanism breaks down.
  • Intermediate Diversity: This is the optimal state. Tastes are different enough that some agents will naturally explore new items even when a popular choice exists (because their subjective score for the popular choice might be low). Crucially, because there is still a common quality component, these individual explorations benefit the entire society by uncovering high-quality items for everyone.

Simulation and Asymptotic Results

The authors provide both mathematical proofs and numerical simulations to support these findings. A key theoretical result is that while utility is bounded by a constant for the extreme cases ($\sigma^2=0$ and $\sigma^2=1$), it grows at a rate of roughly $\sqrt{\ln(\ln T)}$ for intermediate diversity, where $T$ is the number of agents.

Average utility over 100 rounds for different diversity levels. Plot (a) shows results for search cost $c=0.1$, and plot (b) for $c=0.2$. Intermediate diversity levels (red and blue lines) eventually outperform the homogeneous case (yellow line).

As shown in Figure 1, the "no diversity" group (yellow line) initially achieves higher utility because they immediately exploit the first good item found. However, they quickly plateau. In contrast, groups with moderate diversity (e.g., $\sigma^2 = 0.5$, red line) continue to improve as they discover better items over time. The "full diversity" group (purple line) remains stagnant at a low utility level, reflecting the lack of social learning.

Implications and Future Directions

The study highlights that diversity is not just a social ideal but a functional engine for discovery in decentralized systems. It suggests that platforms should value and even highlight diverse perspectives to prevent information stagnation.

Future research could explore "revealed value" models where agents only observe the total utility of their predecessors, rather than the isolated quality component. Additionally, investigating how varying search costs—such as the difference between expert and novice searchers—interacts with diversity could provide deeper insights into the dynamics of social learning.

Summary updated

Abstract

In consumer search, there is a set of items. An agent has a prior over her value for each item and can pay a cost to learn the instantiation of her value. After exploring a subset of items, the agent chooses one and obtains a payoff equal to its value minus the search cost. We consider a sequential model of consumer search in which agents' values are correlated and each agent updates her priors based on the exploration of past agents before performing her search. Specifically, we assume the value is the sum of a common-value component, called the quality, and a subjective score. Fixing the variance of the total value, we say a population is more diverse if the subjective score has a larger variance. We ask how diversity impacts average utility. We show that intermediate diversity levels yield significantly higher social utility than the extreme cases of no diversity (when agents under-explore) or full diversity (when agents are unable to learn from each other) and quantify how the impact of the diversity level changes depending on the time spent searching.
BibTeX
@inproceedings{immorlica2019diversity,
  author = {Nicole Immorlica and Jieming Mao and Christos Tzamos},
  title = {{Diversity and Exploration in Social Learning}},
  booktitle = {The Web Conference},
  year = {2019},
  url = {https://openreview.net/forum?id=B1b58GWdbH},
}
Figure