Introduction: The Accountability Gap
In the era of the GDPR and the "right to explanation," a critical gap has emerged between algorithmic transparency and accountability. While transparency allows us to see the code or weights behind a decision, accountability requires that a human decision-maker can "own" and defend a recommendation to stakeholders.
Most Multi-Criteria Decision Analysis (MCDA) tools rely on "black-box" artifacts like numerical weights or scoring functions. Explaining a job candidate's rejection based on a weighted average is often unpersuasive. This paper introduces a multicriteria ordinal sorting model designed for accountability, allowing for human-readable arguments that mirror natural reasoning.
The Proposed Model: Non-Compensatory Ordinal Sorting
The authors propose a model built on four core design principles to ensure accountability:
- Indirect Elicitation: The system learns from "reference objects"—examples already categorized by the user—rather than asking for abstract weights.
- Robustness: Recommendations are valid across all possible parameters consistent with the user's examples.
- Non-Compensation: A high performance in one area cannot mask a failure in another. This mimics strict human requirements (e.g., a car is not "Safe" if it lacks airbags, regardless of its braking performance).
- No Frontiers: The model avoids arbitrary numerical thresholds, relying instead on direct comparisons to reference objects.
The technical core involves identifying Sufficient Coalitions of criteria. If a user ranks Object A higher than Object B because it performs better on criteria {speed, cost}, the system identifies this subset as "sufficient" to justify the ranking.
Illustrative Dataset
To demonstrate the model, the authors use a synthetic dataset with four criteria ($a, b, c, d$) and six reference objects ($A_1, A_2, B_1, B_2, C_1, C_2$) pre-assigned to three ordered classes (\star, \star\star, \star\star\star).
Two new candidates, $X$ and $Y$, are introduced to show how the model handles sorting and potential "incomparability."
Key Insights: From Trust to Accountability
A central argument of the paper is that while trust relies on accuracy, accountability relies on persuasion. A model is accountable only if its logic can be transferred from the machine to the human.
The model utilizes the "power of impossibility," focusing on proving why an object cannot belong to certain classes. This "burden of proof" approach makes the logic sturdier. Furthermore, by treating criteria as purely ordinal (better vs. worse) rather than cardinal (numerical values), the model avoids the jargon of numerical scaling that often confuses non-experts.
Results and Argumentation
The paper formalizes several argument schemes, such as OUTRANKING and WEAK DOMINANCE, which can be automatically populated to explain a decision.
As shown in the analysis above, the model doesn't always provide a single class; it may provide an interval of possible classes. For instance, it might conclude a candidate is "at least \star\star but no higher than \star\star\star." It also identifies conflicts, such as when a user's initial examples are inconsistent, providing a vital feedback loop for decision-makers.
Limitations and Future Work
Current limitations include the handling of inconsistent preference information and computational complexity as the number of criteria grows. Future work aims to move toward "dialogue games" where users and systems can debate the importance of specific criteria, further enhancing the accountability of the final decision.
Summary updated