Leveraging Tropical Algebra to Assess Trustworthy AI

Juliette Mattioli, Martin Gonzalez, Lucas Mattioli, Karla Quintero, Henri SohierAAAI 2024 · DOI

Abstract

Given the complexity of the application domain, the qualitative and quantifiable nature of the concepts involved, the wide heterogeneity and granularity of trustworthy attributes, and in some cases the non-comparability of the latter, assessing the trustworthiness of AI-based systems is a challenging process. In order to overcome these challenges, the Confiance.ai program proposes an innovative solution based on a Multi-Criteria Decision Aiding (MCDA) methodology. This approach involves several stages: framing trustworthiness as a set of well-defined attributes, exploring attributes to determine related Key Performance Indicators (KPI) or metrics, selecting evaluation protocols, and defining a method to aggregate multiple criteria to estimate an overall assessment of trust. This approach is illustrated by applying the RUM methodology (Robustness, Uncertainty, Monitoring) to ML context, while the focus on aggregation methods are based on Tropical Algebra.

Introduction

Assessing AI trustworthiness is often a "comparing apples to oranges" problem because metrics like robustness, fairness, and accuracy use different scales and units. In the push for "Trustworthy AI," researchers have moved far beyond simple accuracy to include robustness (resistance to noise), uncertainty (the model knowing when it’s guessing), and monitoring (detecting drift at runtime).

Standard approaches usually rely on a weighted sum, but this is often misleading. Weighted sums assume that all criteria are independent and that a high score in one area can perfectly compensate for a catastrophic failure in another. In safety-critical systems, such as autonomous driving or medical AI, a "good average" is insufficient if the system fails a single critical robustness test.

Methodology

The proposed framework utilizes a four-step methodology based on Multi-Criteria Decision Aiding (MCDA), specifically leveraging the RUM (Robustness, Uncertainty, Monitoring) framework to structure the problem.

Overview of the four-stage methodology for assessing AI trustworthiness, from attribute definition to final assessment.

The process follows these stages:

  1. Attribute Definition: Breaking "trust" into specific, measurable traits based on existing taxonomies and standards.
  2. Semantic Structuring: Organizing these traits into a hierarchical tree where the "leaves" are raw KPIs and the "root" is the final trust score.
  3. Commensurability: Normalizing every metric into a "satisfaction scale" from 0 to 1 to allow for comparison.
  4. Tropical Aggregation: Aggregating diverse KPIs into a single, meaningful score using Tropical Algebra.

Tropical Algebra for Aggregation

The core innovation lies in the use of Tropical Algebra (specifically $min$-plus or $max$-plus algebra) for aggregation. In a $(min, +)$ system, the "addition" operation is replaced by taking the minimum:

  • Tropical Addition: $a \oplus b = min(a, b)$
  • Tropical Multiplication: $a \otimes b = a + b$

By using these operations, the framework can create "worst-case" aggregators. If a model is brilliant at everything but fails miserably under one specific type of attack, the Tropical $min$ operator ensures the final score reflects that critical weakness rather than hiding it in an average.

Experimental Evaluation

The paper demonstrates its approach using a "toy use-case" based on the MNIST dataset of handwritten digits. While MNIST is a standard benchmark, it is used here to simulate various real-world stressors.

Visual representation of the MNIST dataset under clean conditions and various stressors, including adversarial attacks and environmental noise.

The evaluation includes:

  • Adversarial Attacks: FGSM (Fast Gradient Sign Method) and PGD (Projected Gradient Descent).
  • Physical/Environmental Perturbations: Patch attacks and Gaussian blur (simulating environmental noise).

Results and Discussion

An MNIST classifier was evaluated across five conditions. The raw accuracy results were:

  • Clean: 0.99
  • FGSM Attack: 0.77
  • PGD Attack: 0.44
  • Patch Attack: 0.94
  • Environmental Noise: 0.96

Using the Tropical aggregator ($\mu_R$), global robustness scores were calculated for different scenarios. In a Worst-Case In-Distribution scenario, the PGD attack (the lowest score) dominated the result, leading to a global score of 0.34.

This single number provides a clear, mathematically grounded signal to AI scientists: the PGD attribute is the primary bottleneck. The Tropical approach naturally highlights the "leading attribute" causing a lack of trust, acting as a mathematical spotlight on the weakest link in the AI's defense.

Conclusion

The RUM methodology, combined with Tropical Algebra, offers a rigorous way to assess AI trustworthiness. By moving away from simple weighted sums, engineers can better identify critical vulnerabilities. Future work will focus on scaling this evaluation to complex, high-dimensional industrial AI systems and expanding the framework to cover AI paradigms beyond Machine Learning, creating a more universal reliability assessment framework.

Summary updated

Abstract

Given the complexity of the application domain, the qualitative and quantifiable nature of the concepts involved, the wide heterogeneity and granularity of trustworthy attributes, and in some cases the non-comparability of the latter, assessing the trustworthiness of AI-based systems is a challenging process. In order to overcome these challenges, the Confiance.ai program proposes an innovative solution based on a Multi-Criteria Decision Aiding (MCDA) methodology. This approach involves several stages: framing trustworthiness as a set of well-defined attributes, exploring attributes to determine related Key Performance Indicators (KPI) or metrics, selecting evaluation protocols, and defining a method to aggregate multiple criteria to estimate an overall assessment of trust. This approach is illustrated by applying the RUM methodology (Robustness, Uncertainty, Monitoring) to ML context, while the focus on aggregation methods are based on Tropical Algebra.
BibTeX
@inproceedings{mattioli2024leveraging,
  author = {Juliette Mattioli and Martin Gonzalez and Lucas Mattioli and Karla Quintero and Henri Sohier},
  title = {{Leveraging Tropical Algebra to Assess Trustworthy AI}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2024},
  doi = {10.1609/aaaiss.v4i1.31775},
}
Figure