WHoW: A Cross-domain Approach for Analysing Conversation Moderation

Ming-Bin Chen, Lea Frermann, Jey Han LauNAACL 2025 · DOI

Abstract

Moderation in multi-party discussions is a constructive conversational act that balances participation, elicits information, and maintains social harmony. We introduce the WHoW framework (Why, How, Who) to provide a structured taxonomy for analyzing moderator interventions across different domains. By applying this framework to TV debates and radio panels, we demonstrate that moderation styles are deeply context-dependent. Our findings show that debate moderators primarily function as "coordinators" enforcing formal rules, while panel moderators act as "contributors" who actively advance the topic. This work utilizes a large-scale dataset annotated by GPT-4o, establishing a foundation for future AI-driven facilitation agents.

Introduction: Beyond "Janitorial" Moderation

Most contemporary research on moderation focuses on the "janitorial" aspects of online spaces—identifying toxic comments, banning bad actors, or filtering spam. However, in live, multi-party settings like TV debates, group therapy sessions, or classrooms, moderation is a proactive and constructive act. Moderators must navigate complex social dynamics, ensure equitable participation, and guide the flow of information in real-time.

Despite its critical role in healthy discourse, there has been a lack of a unified, cross-domain framework to analyze these positive facilitation strategies. Existing studies are often limited by small sample sizes or domain-specific labels that do not translate well between different conversational contexts. The WHoW framework addresses this gap by providing a generalizable taxonomy and a large-scale annotated dataset.

The WHoW Framework and Taxonomy

The authors propose the WHoW framework, which deconstructs every moderator intervention into three distinct dimensions: Why (Motives), How (Dialogue Acts), and Who (Target Speaker).

Table 1: Definitions of the WHoW dimensions and their respective labels, providing a standardized vocabulary for moderation analysis.

Dimensions of Moderation

  1. Motives (Why): This dimension identifies the underlying goal of the intervention. Is the moderator's intent Informational (advancing the topic), Coordinative (enforcing rules and time constraints), or Social (managing the atmosphere and interpersonal dynamics)?
  2. Dialogue Acts (How): This describes the specific conversational tool employed. Labels include Probing (asking questions), Confronting (pitting speakers against each other), Instruction (explicit commands), Interpretation (summarizing or reframing), and Supplement (adding information).
  3. Target Speaker (Who): This identifies the recipient of the intervention, whether it is a specific individual, a team, the audience, or the entire group.
Figure 1: An illustrative example of the WHoW framework in action during a debate on the minimum wage, showing how moderator utterances are mapped to motives, dialogue acts, and targets.

Methodology and Datasets

To test the framework's generalizability, the researchers utilized two primary datasets representing different professional media environments:

  • DEBATE: Transcripts from Intelligent Squared US, a formal TV debate show characterized by strict Oxford-style rules and clear adversarial positions.
  • PANEL: Transcripts from NPR’s Roundtable, a radio program featuring more fluid, conversational, and collaborative discussions.

The study produced a dataset of 5,657 human-annotated sentences and a massive 15,494 GPT-4o annotated sentences. To scale the annotation process, the team compared two prompting strategies for the LLM: Single-Task (ST), predicting one dimension at a time, and Multi-Task (MT), predicting all three simultaneously. The Multi-Task approach generally performed better, as it allowed the model to capture the inherent overlaps between motives and dialogue acts.

Key Insights: Coordinator vs. Contributor

The analysis reveals that moderation is not one-size-fits-all and is heavily influenced by the conversational domain:

  • Moderators as Coordinators: In the DEBATE domain, 66% of moderator motives are coordinative. They rely heavily on instructions and "confronting" dialogue acts—explicitly prompting one speaker to respond to another's point—to maintain the formal structure and foster direct interaction.
  • Moderators as Contributors: In the PANEL domain, 72% of motives are informational. Panel moderators spend significantly more time supplementing the conversation with their own knowledge and addressing speakers individually, resulting in a more personalized, interview-like style.
  • Speaker Dynamics: In both domains, moderators are most likely to intervene after a speaker has held the floor for more than one turn. However, interventions in PANEL discussions are more likely to lead to a "rotation" (a change in speaker), whereas DEBATE moderators often follow up with the same person to clarify a point.

Results and LLM Performance

The researchers found that human agreement (Krippendorff’s alpha) was higher for the PANEL dataset (0.54–0.75) than the DEBATE dataset (0.37–0.72), reflecting the higher complexity of formal debates. GPT-4o achieved moderate-to-good agreement with human judges, with Macro-F1 scores ranging from 0.46 to 0.76. While the model occasionally struggled with subtle distinctions—such as the difference between a coordinative "stop" signal and a social "okay"—it proved to be a viable tool for large-scale behavioral analysis.

Conclusion and Future Work

The WHoW framework provides a "foundational skeleton" for understanding and eventually generating effective moderation. While this study focused on professional media, future work should extend the framework to more diverse settings like group counseling or educational environments. Ultimately, the authors envision using this framework to train AI moderator agents that can facilitate healthier, more productive human discussions across the digital and physical world.

Summary updated

BibTeX
@inproceedings{chen2025whow,
  author = {Ming-Bin Chen and Lea Frermann and Jey Han Lau},
  title = {{WHoW: A Cross-domain Approach for Analysing Conversation Moderation}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2025},
  doi = {10.18653/v1/2025.naacl-long.105},
}

Loading related content...

Figure