Deliberation as Evidence Disclosure: A Tale of Two Protocol Types

Julian Chingoma, Adrian HaretIJCAI 2023 · DOI

TL;DR

This paper explores how structured communication, specifically evidence disclosure, can improve group decision-making. They introduce a formal model and analyze two types of deliberation protocols (simultaneous and sequential) to understand when groups can reliably select the objectively best option, even with unevenly distributed information.

What Problem Does This Paper Address?

The paper tackles the long-standing question of whether groups can be "wise" – meaning they can collectively arrive at the correct decision. While classical models often assume independent agents, real-world decision-making frequently involves deliberation and communication. The authors are interested in whether structured information exchange, formalized as evidence disclosure, can lead to better outcomes than individual decision-making, particularly when the evidence supporting different options is not equally distributed among the agents. They aim to identify conditions under which deliberation protocols can guarantee that the objectively best alternative is chosen.

Methods

The authors propose an "evidence-based election model" where agents have access to objective evidence supporting various alternatives. Deliberation is modeled as a protocol where agents iteratively disclose evidence and update their beliefs. They distinguish between two main types of protocols:

  • Simultaneous Protocol (Psim): Agents disclose evidence at the same time, and then all agents update their knowledge and rankings. This is likened to a shared discussion board.
  • Sequential Protocols: Agents take turns disclosing evidence. Two variants are introduced:
  • Sequential Constant Protocol (Pseq-con): Agents disclose evidence based on the current plurality winner of nominations.
  • Sequential Abstention Protocol (Pseq-abs): Agents can choose to abstain from nominating or disclosing if they don't dissent, and nominations persist across turns.

They analyze these protocols in terms of termination time and, crucially, the conditions under which they lead to the selection of the optimal alternative. The analysis considers two agent types: "lazy" (less proactive in disclosing) and "keen" (more proactive).

Datasets

Not discussed in the paper. The paper focuses on a theoretical model and uses simulated scenarios for evaluation.

Key Insights and Ideas

  • Deliberation isn't a silver bullet: Even with objective evidence, groups can still make suboptimal decisions. This can happen when a small, vocal group with evidence for an inferior option can sway the majority.
  • Evidence gap matters: A significant advantage in evidence for the optimal alternative is a strong predictor of successful deliberation.
  • Evidence spread is crucial: Unbalanced distributions of evidence (where a few agents hold a lot of evidence for one option) are more likely to lead to poor outcomes than evenly distributed evidence. This suggests that diversity in evidence ownership can be beneficial.
  • Protocol design is key: The specific rules of deliberation (simultaneous vs. sequential, turn order, abstention rules) significantly impact the outcome. Sequential protocols, in particular, can be sensitive to the order in which agents speak.
  • Agent types influence dynamics: "Keen" agents, who are more eager to disclose information, can behave differently from "lazy" agents, affecting the deliberation process.

Key Results

  • Theoretical Guarantees: Under specific conditions (e.g., large evidence gap, complete and disjoint evidence distributions), theoretical guarantees can be made for all protocols to select the optimal alternative. For instance, if the optimal alternative has at least n times more evidence than the second-best, and agents are lazy, the optimal choice is likely.
  • Simultaneous Protocol (Psim): Shows promise, especially for two alternatives, where it can guarantee the optimal choice even with a smaller evidence gap if the optimal alternative is initially an "underdog."
  • Sequential Protocols (Pseq-con, Pseq-abs): Their success is highly dependent on the agent ordering. Pseq-abs, in particular, offers strong guarantees for two alternatives with keen agents, regardless of ordering. However, for more than two alternatives or with lazy agents, guarantees become harder to achieve.
  • Simulation Findings:
    • Success rates increase with the evidence gap.
  • Unbalanced evidence distributions (high variance) lead to lower success rates and longer deliberation times, especially when the evidence gap is small.
  • Deliberation often terminates quickly (around one round) when outcomes are poor, suggesting that agents quickly converge on a suboptimal choice.

Limitations and Future Work

  • Simplified Model: The model assumes objective, unassailable evidence and does not account for agents acquiring evidence or rejecting it.
  • Two Alternatives: Many theoretical results are strongest for the case of two alternatives, with generalizations to more alternatives being more complex.
  • Agent Types: The distinction between "lazy" and "keen" agents is a simplification of real-world agent motivations.
  • Future Work: The authors suggest exploring models where agents can reject evidence, incorporating probabilistic evidence acquisition, and restricting communication to local neighborhoods. They also leave open the question of whether agent orderings can guarantee full-disclosure equivalence for keen agents with neutral tendencies.

Summary updated

BibTeX
@inproceedings{chingoma2023deliberation,
  author = {Julian Chingoma and Adrian Haret},
  title = {{Deliberation as Evidence Disclosure: A Tale of Two Protocol Types}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2023},
  doi = {10.24963/ijcai.2023/288},
}

Loading related content...

Figure