Abstract
Given the complexity of the application domain, the qualitative and quantifiable nature of the concepts involved, the wide heterogeneity and granularity of trustworthy attributes, and in some cases the non-comparability of the latter, assessing the trustworthiness of AI-based systems is a challenging process. In order to overcome these challenges, the Confiance.ai program proposes an innovative solution based on a Multi-Criteria Decision Aiding (MCDA) methodology. This approach involves several stages: framing trustworthiness as a set of well-defined attributes, exploring attributes to determine related Key Performance Indicators (KPI) or metrics, selecting evaluation protocols, and defining a method to aggregate multiple criteria to estimate an overall assessment of trust. This approach is illustrated by applying the RUM methodology (Robustness, Uncertainty, Monitoring) to ML context, while the focus on aggregation methods are based on Tropical Algebra.
Introduction
Assessing AI trustworthiness is often a "comparing apples to oranges" problem because metrics like robustness, fairness, and accuracy use different scales and units. In the push for "Trustworthy AI," researchers have moved far beyond simple accuracy to include robustness (resistance to noise), uncertainty (the model knowing when it’s guessing), and monitoring (detecting drift at runtime).
Standard approaches usually rely on a weighted sum, but this is often misleading. Weighted sums assume that all criteria are independent and that a high score in one area can perfectly compensate for a catastrophic failure in another. In safety-critical systems, such as autonomous driving or medical AI, a "good average" is insufficient if the system fails a single critical robustness test.
Methodology
The proposed framework utilizes a four-step methodology based on Multi-Criteria Decision Aiding (MCDA), specifically leveraging the RUM (Robustness, Uncertainty, Monitoring) framework to structure the problem.
The process follows these stages:
- Attribute Definition: Breaking "trust" into specific, measurable traits based on existing taxonomies and standards.
- Semantic Structuring: Organizing these traits into a hierarchical tree where the "leaves" are raw KPIs and the "root" is the final trust score.
- Commensurability: Normalizing every metric into a "satisfaction scale" from 0 to 1 to allow for comparison.
- Tropical Aggregation: Aggregating diverse KPIs into a single, meaningful score using Tropical Algebra.
Tropical Algebra for Aggregation
The core innovation lies in the use of Tropical Algebra (specifically $min$-plus or $max$-plus algebra) for aggregation. In a $(min, +)$ system, the "addition" operation is replaced by taking the minimum:
- Tropical Addition: $a \oplus b = min(a, b)$
- Tropical Multiplication: $a \otimes b = a + b$
By using these operations, the framework can create "worst-case" aggregators. If a model is brilliant at everything but fails miserably under one specific type of attack, the Tropical $min$ operator ensures the final score reflects that critical weakness rather than hiding it in an average.
Experimental Evaluation
The paper demonstrates its approach using a "toy use-case" based on the MNIST dataset of handwritten digits. While MNIST is a standard benchmark, it is used here to simulate various real-world stressors.
The evaluation includes:
- Adversarial Attacks: FGSM (Fast Gradient Sign Method) and PGD (Projected Gradient Descent).
- Physical/Environmental Perturbations: Patch attacks and Gaussian blur (simulating environmental noise).
Results and Discussion
An MNIST classifier was evaluated across five conditions. The raw accuracy results were:
- Clean: 0.99
- FGSM Attack: 0.77
- PGD Attack: 0.44
- Patch Attack: 0.94
- Environmental Noise: 0.96
Using the Tropical aggregator ($\mu_R$), global robustness scores were calculated for different scenarios. In a Worst-Case In-Distribution scenario, the PGD attack (the lowest score) dominated the result, leading to a global score of 0.34.
This single number provides a clear, mathematically grounded signal to AI scientists: the PGD attribute is the primary bottleneck. The Tropical approach naturally highlights the "leading attribute" causing a lack of trust, acting as a mathematical spotlight on the weakest link in the AI's defense.
Conclusion
The RUM methodology, combined with Tropical Algebra, offers a rigorous way to assess AI trustworthiness. By moving away from simple weighted sums, engineers can better identify critical vulnerabilities. Future work will focus on scaling this evaluation to complex, high-dimensional industrial AI systems and expanding the framework to cover AI paradigms beyond Machine Learning, creating a more universal reliability assessment framework.
Summary updated