Introduction
As artificial intelligence systems are increasingly deployed in high-stakes domains such as healthcare, finance, and criminal justice, the need for robust bias detection has become paramount. However, existing fairness tools often operate as "black boxes," providing numerical scores without explaining the underlying reasons for a model's biased behavior. This lack of transparency hinders trust and complicates mitigation efforts.
To address this, we present ABIDE (Argumentative BIas detection by DEbate), a framework that transforms bias detection into a structured, human-readable debate. By leveraging formal argumentation theory, ABIDE provides faithful justifications for its findings, moving beyond global statistics to uncover local biases within specific sub-groups.
Methodology: Bias Detection as Argumentation
ABIDE structures the detection process as a Quantitative Bipolar Argumentation Framework (QBAF). The system identifies "neighborhoods" of similar individuals and evaluates whether protected groups are disadvantaged relative to their peers.
The core innovation lies in the use of Critical Questions (CQs). Rather than blindly accepting a bias claim, the framework subjects it to "attacks" based on data quality metrics:
- Significance: Is the neighborhood large enough to provide a statistically meaningful comparison?
- Diversity: Does the neighborhood contain a sufficient mix of individuals to ensure a fair baseline?
- Objectivity: Is the neighborhood mathematically well-defined (e.g., convex)?
The final output is a numerical strength value for the presence of bias, derived from the interaction of these supporting and attacking arguments.
Transparent Justification through Debate
A key feature of ABIDE is its ability to present these complex logical structures as a natural language dialogue. This allows auditors to follow the reasoning process step-by-step, ensuring that the explanation is faithful to the underlying model logic.
By presenting the detection results as a debate, the final decision is not a post-hoc interpretation but a direct result of the arguments weighed during the process. This structure naturally handles statistical uncertainty without needing complex hybrid probabilistic models.
Experimental Results
The framework was evaluated using synthetic models with known biases and real-world datasets, including COMPAS and the Adult Census Income (ACI) dataset.
- Enhanced Sensitivity: On the COMPAS recidivism dataset, ABIDE identified 77 instances of bias against African-American individuals, a significant improvement over the baseline IRB method, which detected only 2.
- LLM Auditing: When applied to ChatGPT-4o, ABIDE uncovered 129 instances of racial bias and 6 instances of gender bias that were previously undetected by baseline methods.
- Efficiency and Robustness: ABIDE maintained high precision and recall across varying neighborhood sizes ($K=50$ to $200$) and demonstrated runtimes nearly ten times faster than previous argumentative baselines (e.g., ~4 seconds vs ~48 seconds for certain tests).
Conclusion and Future Directions
ABIDE represents a significant step toward transparent and interpretable AI auditing. By framing bias detection as a debate, it provides stakeholders with the "why" behind fairness metrics. Future work will explore human-in-the-loop scenarios, allowing auditors to interactively challenge the system's arguments, and expanding the framework to support multi-agent consensus building.
Summary updated