Aligning AI with Public Values: Deliberation and Decision-Making for Governing Multimodal LLMs in Political Video Analysis

Tanusree Sharma, Yujin Potter, Zachary Kilhoffer, Yun Huang, Dawn Song, Yang Wang

TL;DR

This paper introduces Inclusive.AI, a decentralized platform that uses blockchain and Decentralized Autonomous Organization (DAO) voting mechanisms to let the public collectively govern how multimodal LLMs interpret sensitive political content. The study highlights a stark contrast between expert and public preferences for AI behavior and demonstrates that quadratic voting with equal power distribution significantly enhances the perceived fairness and democratic legitimacy of AI alignment.

What Problem Does This Paper Address?

As multimodal Large Language Models (LLMs) are increasingly deployed to analyze, summarize, and interpret politically sensitive media (like presidential debates), they bring inherent risks of bias, misinformation, and lack of transparency.

Traditional AI alignment relies heavily on top-down engineering interventions or static, one-off feedback methods like surveys and interviews. These approaches fail to capture the nuanced, ongoing deliberation required to navigate contentious societal topics. This paper addresses a critical gap: How can we build a scalable, transparent, and democratic framework that allows diverse public values to directly guide and govern AI model behavior?

Methods

The authors developed Inclusive.AI, a democratic governance platform built on the Optimism blockchain. The platform leverages smart contracts and DAO infrastructure to ensure secure, transparent, and pseudonymous participation.

The research followed a two-step methodology:

  1. Expert Baseline: The authors interviewed 10 journalists and media experts to establish a professional baseline for political video analysis and to select a representative "seed case."
  2. Public Deliberation and Voting: A user study was conducted with 114 US participants. The workflow consisted of three phases:
  3. Individual Deliberation: Users chatted with an AI agent to clarify their personal values regarding an AI-generated summary of a political video.
  4. Group Deliberation: Users joined a live chat room to discuss and co-validate their perspectives with other participants.
  5. Democratic Voting: Users voted on four potential updates to the LLM model (e.g., keeping the current model, providing more specific facts, integrating a feedback loop, or analyzing emotions).

To evaluate how different democratic structures affect user perception, the researchers implemented a $2 \times 2$ between-subjects design, varying the voting method (Quadratic Voting vs. Weighted Voting) and the voting power distribution (Equal Power vs. a 20/80 Pareto split simulating the outsized influence of "early adopters").

Figure 1: An overview of processes, (a) interview with experts to select suitable video example; (b) prepare seed case for experiment setup; (c) incorporate seed case into inclusive.AI system for deliberation and preference gathering (i) deliberation human-AI and group (ii) democratic voting process incorporating the voting options from deliberation of general public.

Datasets

The study utilized a curated 6-minute video clip from the 2020 US Presidential Debate (focusing on healthcare policy and Obamacare) as its primary case study. Because this paper focuses on human-centric governance mechanisms rather than training a new model from scratch, no new large-scale machine learning datasets or benchmarks were introduced.

Key Insights and Ideas

  • The Expert-Public Priority Mismatch: There is a fundamental divide in what experts and the lay public want from AI. While media experts emphasized the importance of capturing emotional nuances, dramatic tension, and narrative storytelling, the general public prioritized strict factual clarity, objectivity, and emotional neutrality.
  • Voting Architecture Shapes Alignment: The choice of voting mechanism heavily dictates the trajectory of AI updates. Quadratic voting—where the token cost of casting multiple votes on a single issue increases quadratically—successfully curbed the dominance of majority factions and protected minority preferences.
  • Power Concentration Distorts Public Will: Under the asymmetric 20/80 voting power condition, a small group of influential stakeholders disproportionately altered the voting outcomes. This highlights a real-world risk: if AI governance tokens are concentrated among a few early adopters or wealthy stakeholders, the resulting AI alignment may diverge sharply from broader public values.

Key Results

  • Top Public Preference: Across all experimental conditions, participants strongly preferred updating the LLM to "provide more specific facts" (Choice 2), followed by "analyzing speakers' emotions and sentiment" (Choice 4).
  • Political Leaning Variations: Political affiliation subtly influenced governance choices. Compared to Democrats, Republican participants were less likely to vote for more specific facts ($P = 0.084$) and significantly more likely to favor integrating a continuous user feedback loop ($P = 0.054$).
  • Democratic Legitimacy: Quadratic voting combined with equal power distribution yielded the highest user trust and perceived fairness (measured via the international Varieties of Democracy, or V-Dem, scale). Linear regression confirmed that quadratic voting significantly boosted perceived democratic quality (coefficient = 0.4772, $P = 0.013$), an effect that was amplified when paired with equal power distribution (interaction coefficient = 1.1548, $P = 0.002$).

Limitations and Future Work

  • Implementation Skepticism: Many participants remained skeptical about whether commercial AI companies would actually honor and implement the results of decentralized public votes. The authors note that government-level compliance guidelines and independent audits will be necessary to enforce true accountability.
  • Echo Chamber Risks: Tailoring LLM behaviors to specific group preferences risks reinforcing political biases and potentially deepening ideological divides.
  • Future Scaling: The current study was bounded by a single political video case study. The authors propose scaling Inclusive.AI into a live browser plugin for platforms like ChatGPT, allowing real-time public challenges and collaborative deliberation across other high-stakes domains, such as medical misinformation and legislative summaries.

Summary updated

Abstract

How AI models should deal with political topics has been discussed, but it remains challenging and requires better governance. This paper examines the governance of large language models through individual and collective deliberation, focusing on politically sensitive videos. We conducted a two-step study: interviews with 10 journalists established a baseline understanding of expert video interpretation; 114 individuals through deliberation using InclusiveAI, a platform that facilitates democratic decision-making through decentralized autonomous organization (DAO) mechanisms. Our findings reveal distinct differences in interpretative priorities: while experts emphasized emotion and narrative, the general public prioritized factual clarity, objectivity, and emotional neutrality. Furthermore, we examined how different governance mechanisms - quadratic vs. weighted voting and equal vs. 20/80 voting power - shape users' decision-making regarding AI behavior. Results indicate that voting methods significantly influence outcomes, with quadratic voting reinforcing perceptions of liberal democracy and political equality. Our study underscores the necessity of selecting appropriate governance mechanisms to better capture user perspectives and suggests decentralized AI governance as a potential way to facilitate broader public engagement in AI development, ensuring that varied perspectives meaningfully inform design decisions.
BibTeX
@inproceedings{sharma2025aligning,
  author = {Tanusree Sharma and Yujin Potter and Zachary Kilhoffer and Yun Huang and Dawn Song and Yang Wang},
  title = {{Aligning AI with Public Values: Deliberation and Decision-Making for Governing Multimodal LLMs in Political Video Analysis}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2025},
  doi = {10.1609/aies.v8i3.36720},
}

Loading related content...

Figure