D-Agree: Crowd Discussion Support System Based on Automated Facilitation Agent

Takayuki Ito, Shota Suzuki, Naoko Yamaguchi, Tomohiro Nishida, Kentaro Hiraishi, Kai YoshinoAAAI 2020 · DOI

Abstract

Large-scale online discussions are essential for modern digital democracy but often become unmanageable due to the high volume of participants and the unstructured nature of threads. This paper introduces D-Agree, an automated facilitation system designed to support crowd-scale discussions. By leveraging deep learning to map conversations into a structured Issue-Based Information System (IBIS) framework, D-Agree generates targeted prompts to guide participants. A real-world evaluation involving nearly 800 participants in Nagoya, Japan, shows that the system achieves user satisfaction levels comparable to human facilitators, particularly when used in a hybrid human-agent configuration.

Introduction

As digital platforms become central to urban planning and governance, the need for effective moderation at scale has grown. While human facilitators are effective in small-group settings, they cannot process the thousands of simultaneous posts generated in crowd-scale environments. This leads to "tangled" threads where opinions overlap and productive consensus is hard to reach. D-Agree addresses this by providing an Automated Facilitation Agent that extracts the underlying logic of a discussion and intervenes to maintain focus and depth.

System Architecture and Methods

The D-Agree system operates through a multi-stage pipeline designed to transform raw text into a structured knowledge graph.

Overview of the D-Agree system architecture, illustrating the extraction of IBIS components and the automated facilitation loop.
  1. Discussion Structuring (IBIS): The system utilizes the Issue-Based Information System (IBIS) framework to categorize posts into four distinct types: Issues (questions), Ideas (solutions), Pros, and Cons.
  2. Argumentation Mining: A BiLSTM (Bidirectional Long Short-Term Memory) network is employed to identify these components and their interrelationships within the discussion.
  3. Rule-Based Facilitation: The agent utilizes a library of approximately 200 facilitation rules, developed alongside professional human facilitators.
  4. Automated Intervention: By comparing the current state of the IBIS graph against its rule library, the agent identifies gaps—such as an idea without listed drawbacks—and automatically posts prompts (e.g., "What are the potential downsides to this proposal?") to elicit further detail.

Experimental Evaluation

The system was evaluated through a large-scale social experiment conducted in collaboration with the Nagoya municipal government. The discussion focused on the "Nagoya-city Next-Generation Comprehensive Plan," attracting 798 participants who submitted 432 opinions across five themes.

The experiment compared three facilitation modes:

  • Themes 1 & 2: Human-led facilitation.
  • Themes 3 & 4: Agent-led facilitation (Auto FA).
  • Theme 5: Hybrid facilitation (Human + Agent).

Results and Key Insights

The evaluation focused on user satisfaction and the productivity of the discussion.

Average user satisfaction scores across the five experimental themes, comparing human, automated, and hybrid facilitation.
  • Comparable Satisfaction: The Automated Facilitation Agent achieved satisfaction scores between 3.2 and 3.6 on a 5-point scale, statistically comparable to the 3.4 score of human facilitators.
  • The Hybrid Advantage: The highest satisfaction (3.7) was recorded in Theme 5, where humans and AI worked together. This suggests that while AI provides consistent, tireless prompting, human oversight provides the high-level strategic guidance necessary for optimal results.
  • Productivity Gains: In the hybrid setup, the generation of "Ideas" and "Pros" per facilitation message increased significantly. According to the experimental data, the performance in generating "Ideas" was 280.6% higher than in other themes, indicating that the agent is highly effective at eliciting specific content types when guided by a human moderator.

Conclusion

D-Agree demonstrates that automated agents can effectively manage large-scale public discourse by imposing structure on unstructured data. By acting as a "deepener" rather than just a moderator, the system moves crowds from vague suggestions toward concrete, well-vetted proposals. Future work will focus on refining the facilitation rules for more polarized topics and enhancing the system's ability to handle toxic behavior or "flaming" within the IBIS framework.

Summary updated

BibTeX
@inproceedings{ito2020d,
  author = {Takayuki Ito and Shota Suzuki and Naoko Yamaguchi and Tomohiro Nishida and Kentaro Hiraishi and Kai Yoshino},
  title = {{D-Agree: Crowd Discussion Support System Based on Automated Facilitation Agent}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2020},
  doi = {10.1609/aaai.v34i09.7094},
}
Figure