Putting Accountability of AI Systems into Practice

Beatriz San Miguel, Aisha Naseer, Hiroya InakoshiIJCAI 2020 · DOI

TL;DR

This paper introduces the Global-view Accountability Framework (GAF), a novel system designed to translate abstract AI accountability principles into practical technical solutions. GAF addresses the challenge of conflicting information from multiple AI systems by auditing data, identifying discrepancies, and generating a unified, corrected output to prevent negative impacts, demonstrated here for autonomous vehicles and insurance claims.

What Problem Does This Paper Address?

The increasing integration of Artificial Intelligence (AI) systems into critical domains like autonomous vehicles and insurance necessitates robust accountability mechanisms. While ethical AI guidelines and principles abound, translating these into concrete, implementable technical solutions remains a significant challenge. A key aspect of accountability is ensuring that AI systems operate responsibly, are auditable, and can mitigate negative consequences. This paper tackles the problem of how to manage and reconcile potentially conflicting information generated by multiple AI systems operating within the same context, which could otherwise lead to undesirable outcomes, such as accidents or unfair insurance claims.

Methods

The core of the paper is the Global-view Accountability Framework (GAF). GAF is designed to ingest data from multiple AI systems, audit it for discrepancies, and produce a unified, corrected "Global Information" (GI) output. The framework comprises six key components:

  • Information Collector: Gathers and standardizes data from various AI systems.
  • Conflict Auditor: Identifies discrepancies in the received data.
  • Confidence Score Generator: Assigns a confidence score to each AI system's perception, using a multi-method approach that considers historical data, majority voting, and proximity to other perceptions.
  • GI Activator: Uses the calculated GI to send corrective orders to AI systems with erroneous perceptions, aiming to prevent negative impacts.
  • Historical Data Store: Logs all data processed by the GAF for auditing and analysis.
  • Report Generator: Provides relevant data for third parties (e.g., auditors, insurance adjusters) to monitor systems, investigate incidents, or assign liability.

The paper details a specific method for calculating the GI, inspired by barycentric coordinates, which combines the values and confidence scores of individual AI systems' perceptions.

Datasets

Not discussed in the paper. The demonstration uses a simulated environment with vehicles and pedestrians.

Key Insights and Ideas

  • Bridging the Theory-Practice Gap: The GAF is a concrete attempt to operationalize abstract ethical AI principles, moving beyond recommendations to actual technical implementation.
  • Proactive Conflict Resolution: Instead of just reacting to failures, GAF actively audits and reconciles conflicting data before it can cause harm, particularly in scenarios involving multiple interacting AI agents.
  • Dynamic Confidence Scoring: The confidence score is not static; it's updated based on historical performance and consensus, allowing the system to learn and adapt to the reliability of individual AI systems over time.
  • Enabling Third-Party Oversight: The Report Generator is crucial for external stakeholders, providing the necessary audit trails and insights for accountability, liability assessment, and system improvement.

Key Results

The paper demonstrates the GAF's functionality in a simulated automotive scenario where multiple vehicles and a pedestrian are present. The key result is the GAF's ability to detect and correct a misperception of a pedestrian's location by one of the vehicles.

  • Scenario: A pedestrian is partially obscured from one vehicle (car_2) by other vehicles (car_1, car_4).
  • Discrepancy: van_1 perceives the pedestrian at (650, 400), car_1 at (400, 400), while car_3 correctly identifies the position at (600, 400).
  • GAF Action: The Conflict Auditor identifies these discrepancies. The GI calculation, using the confidence scores (initially set to maximum for all vehicles in this initial stage), results in a corrected pedestrian position of (550, 400).
  • Outcome: The GI Activator would then inform car_2 (and potentially others) of this corrected position, increasing its awareness and potentially preventing an accident. The paper notes that with repeated use, confidence scores would adjust, leading to more accurate GIs.

Limitations and Future Work

  • Initial Stage Limitations: The authors acknowledge that in the initial stage, with no historical data, the confidence scores are uniform, which can lead to less accurate GIs (as seen with the pedestrian's position being calculated as 550 instead of the actual 600). They emphasize that confidence scores will improve with frequent updates and executions.
  • Simulated Environment: The current testing and validation are performed in a simulated environment.
  • Future Work: The authors plan to trial the GAF in real-world scenarios for evaluation. They also mention that while the demonstration focused on pedestrian position discrepancies, the framework can be extended to other features.

Summary updated

BibTeX
@inproceedings{miguel2020putting,
  author = {Beatriz San Miguel and Aisha Naseer and Hiroya Inakoshi},
  title = {{Putting Accountability of AI Systems into Practice}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2020},
  doi = {10.24963/ijcai.2020/768},
}
Figure