Power Echoes: Investigating Moderation Biases in Online Power-Asymmetric Conflicts

Yaqiong Li, Peng Zhang, Peixu Hou, Kainan Tu, Guangping Zhang, Shan Qu, Wenshi Chen, Yan Chen, Ning Gu, Tun LuarXiv 2026 · arXiv

This paper explores how human and AI-assisted moderators handle conflicts where one party holds significantly more power than the other—such as a merchant versus a consumer. The researchers reveal that human moderators naturally lean toward the powerful party when presented with specific power cues, and while AI assistance can mitigate some of these biases, it can also inadvertently amplify others through a "legitimizing" effect.

TL;DR

Human moderators exhibit significant biases toward the powerful party in online conflicts, often using power cues as heuristics for credibility. While AI assistance alleviates most of these biases by refocusing moderators on logical contradictions, it can also introduce new biases by "legitimizing" authoritative claims made by the powerful party.

What Problem Does This Paper Address?

Online platforms like Yelp, Amazon, and Dianping are hotbeds for conflicts between users with unequal standing. These power-asymmetric conflicts typically involve one party with more resources and expertise (e.g., a merchant) and another who relies on the platform for coordination (e.g., a consumer).

While we know human moderators are prone to general biases (like those related to identity or toxicity), we don't yet understand how they react to structural power imbalances. Furthermore, as platforms increasingly adopt AI-assisted moderation, it is unclear whether these tools help level the playing field or simply reinforce existing power structures.

Methods

The authors conducted a mixed-design experiment with 50 participants using a custom web-based program called "I Support." Their approach involved:

  1. Taxonomy Development: Based on the "Bases of Social Power" theory, they identified 10 "power manifestations" relevant to online conflicts, such as citing platform rules (legitimate power) or offering refunds (reward power).
  2. Perturbation Analysis: They took real conflict samples and created "perturbed" versions by inserting specific power cues into either the consumer's or merchant's statement to see how it shifted moderator judgment.
  3. Wizard-of-Oz AI: To isolate the effect of perceived AI assistance without the noise of LLM hallucinations or tone variations, researchers provided high-quality, human-crafted suggestions but told participants they were generated by an AI.
  4. Mixed Design: Participants were split into a "Human Moderation" group (no AI) and a "Human-AI Moderation" group, while also seeing both initial and perturbed conflict samples.
Figure 1. The flow of experimental design.

Datasets

The researchers leveraged data from Dianping, a leading Chinese local life service platform (similar to Yelp).

  • Initial Corpus: ~60,000 consumer-merchant conflict samples.
  • Selected Set: 100 representative samples across five top categories: Food, Beauty, Entertainment, Sports, and Hotels.
  • Experimental Set: 2,000 samples (100 initial + 1,900 perturbed versions) were generated to test the 10 power manifestations.

Key Insights and Ideas

  • The "Powerful Party" Bias: Human moderators consistently favor the merchant when power cues are present. Interestingly, even when a consumer uses "expert knowledge," moderators often view it as "intentional ornamentation," whereas the same jargon from a merchant is seen as a "credible explanation."
  • Heuristics over Facts: Moderators often use surface-level cues—like the length of a response or the citation of a rule—as shortcuts for credibility rather than deeply analyzing the conflict's core facts.
  • AI as a Logical Anchor: AI assistance was most effective when it highlighted logical flaws in the opposing party's statement. This "unsupporting" perspective was more powerful at shifting support toward the weaker party than simply listing reasons to support them.
  • The Legitimizing Effect: AI can be a double-edged sword. When an AI suggestion included a rule citation, moderators were even more likely to support the merchant, viewing the AI's mention as a "completed verification" of the merchant's claim.
Table 1: Ten forms of power manifestation in online power-asymmetric conflicts.

Key Results

  • Human Bias: Human moderators showed significant bias toward the powerful party in 5 out of 10 manifestations: legitimate claims, punishment threats, compensation, expert knowledge, and length difference.
  • AI Mitigation: AI assistance successfully alleviated four of these biases (punishment threat, compensation, expert knowledge, and length) and completely eliminated one.
  • AI Amplification: AI introduced a new authority citation bias and amplified the legitimate claim bias. In the latter case, support for the merchant increased by 8.3% compared to human-only moderation.
  • Demographic Variations: Younger participants were more sensitive to logical consistency, while older participants were more influenced by authority and interest-related cues.
Examples of how conflict samples were perturbed to test specific biases.

Limitations and Future Work

The study is limited by its focus on a single platform (Dianping) and a specific cultural context (China). The authors note that moderation biases are heavily influenced by cultural ideology and language, meaning these results might not generalize globally.

Additionally, the use of a Wizard-of-Oz design, while excellent for experimental control, doesn't account for the tone and accuracy issues inherent in real-world LLMs. Future work should explore:

  • Cross-cultural studies to see how power dynamics shift in different societies.
  • The impact of varying levels of "AI literacy" among moderators.
  • Design interventions like "anonymization" (labeling parties as Subject A and B) or "bias reminders" to further neutralize power-related heuristics.

Summary updated

Abstract

Online power-asymmetric conflicts are prevalent, and most platforms rely on human moderators to conduct moderation currently. Previous studies have been continuously focusing on investigating human moderation biases in different scenarios, while moderation biases under power-asymmetric conflicts remain unexplored. Therefore, we aim to investigate the types of power-related biases human moderators exhibit in power-asymmetric conflict moderation (RQ1) and further explore the influence of AI's suggestions on these biases (RQ2). For this goal, we conducted a mixed design experiment with 50 participants by leveraging the real conflicts between consumers and merchants as a scenario. Results suggest several biases towards supporting the powerful party within these two moderation modes. AI assistance alleviates most biases of human moderation, but also amplifies a few. Based on these results, we propose several insights into future research on human moderation and human-AI collaborative moderation systems for power-asymmetric conflicts.
BibTeX
@misc{li2026power,
  author = {Yaqiong Li and Peng Zhang and Peixu Hou and Kainan Tu and Guangping Zhang and Shan Qu and Wenshi Chen and Yan Chen and Ning Gu and Tun Lu},
  title = {{Power Echoes: Investigating Moderation Biases in Online Power-Asymmetric Conflicts}},
  year = {2026},
  eprint = {2603.01457},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2603.01457},
}
Figure