Abstract
ArguCast is a novel framework and online system that combines human judgmental forecasting with computational argumentation to mitigate cognitive biases. By structuring debates using Quantitative Bipolar Argumentation Frameworks (QBAFs), the system filters irrational user opinions and aggregates predictions more accurately across multiple related events. This paper details the framework's formal structure, rationality constraints, and system implementation.
Introduction
Judgmental forecasting, which leverages human expertise to predict future events, is a critical tool when historical data is insufficient for quantitative models. However, human forecasters are often subject to cognitive biases, overconfidence, and noise. While existing platforms facilitate discussion, they lack a formal structure to evaluate the logical consistency of forecasts. ArguCast addresses these challenges by integrating human judgmental forecasting with computational argumentation, specifically through Quantitative Bipolar Argumentation Frameworks (QBAFs).
The ArguCast Framework (ACF)
The core of the system is the ArguCast Framework (ACF), which structures debates into a formal hierarchy. Unlike traditional systems that treat events in isolation, ACFs allow for multi-forecasting, where arguments can be shared across related questions to reduce data sparsity.
The framework distinguishes between two types of components:
- Forecasting Arguments ($f$): These represent the primary claims or predictions being evaluated (e.g., "LLMs will reach AGI before 2030").
- Non-forecasting Arguments ($d$): These provide the rationales, evidence, or counter-points that support or attack the forecasting arguments.
As shown in the figure above, the system uses "Gradual Semantics" (specifically the DF-QuAD algorithm) to calculate the strength of a claim based on the collective weight of its supporting and attacking arguments.
Rationality Filtering and Aggregation
A key innovation of ArguCast is the introduction of rationality constraints to filter out biased or illogical inputs. The system defines two primary forms of rationality:
- Vote Rationality: A user is considered irrational if they support a claim despite acknowledging that the counter-arguments are stronger.
- Prediction Rationality: A user is considered irrational if their numerical probability assignment (e.g., 80%) contradicts the calculated strength of the arguments they have endorsed.
By identifying and discounting "irrational" forecasts, the system aims to improve the signal-to-noise ratio of the aggregate prediction. The final group forecast is calculated using a weighted geometric mean of odds, where weights are derived from users' historical accuracy as measured by Brier scores.
System Implementation and Discussion
The authors developed a prototype web application using React, Flask, and PostgreSQL. This system provides a visual interface for users to engage in structured debates and submit forecasts.
Key insights from the system design include:
- Argumentation as a Filter: Filtering for logical consistency before aggregation can mitigate the impact of individual biases.
- Transparency: Moving from "black box" intuition to a transparent reasoning tree helps users identify their own cognitive inconsistencies.
- Efficiency: Sharing arguments across multiple forecasting tasks improves the depth of reasoning for complex, interconnected events.
Conclusion and Future Work
ArguCast provides a rigorous mathematical foundation for combining human judgment with structured argumentation. While the current implementation focuses on binary outcomes, the underlying theory supports multi-outcome forecasting. Future research will focus on fully integrating automated rationality filtering into the live system, conducting large-scale empirical studies to validate accuracy improvements, and exploring Explainable AI (XAI) to generate natural language justifications for system-generated forecasts.
Summary updated