A Non-Numeric Approach to Multi-Criteria/Multi-Expert Aggregation Based on Approximate Reasoning

Ronald R. YagerUAI 1991

TL;DR

This paper introduces a novel non-numeric approach for aggregating information from multiple sources and experts in multi-criteria decision-making. By leveraging linguistic scales and approximate reasoning, it bypasses the "tyranny of numbers" and simplifies the evaluation process for experts, offering a more intuitive and manageable way to handle complex decisions.

What Problem Does This Paper Address?

Many real-world problems involve making decisions based on information from various sources, often evaluated by multiple experts. This is particularly true in areas like project funding, medical diagnosis, or information fusion, where a set of alternatives or hypotheses needs to be assessed against several criteria. The challenge lies in effectively combining these diverse inputs, especially when experts have different interpretations or assign varying importance to criteria. Traditional methods often rely on precise numerical scales, which can be burdensome for experts and lead to a false sense of precision. This paper tackles the "multi-criteria/multi-expert aggregation" problem by proposing a method that works with linguistic, non-numeric information.

Methods

The core of the proposed methodology is a two-stage process:

  1. Individual Expert Evaluation: Experts are presented with a linguistic scale (e.g., Perfect, Very High, High, Medium, Low, Very Low, None) to rate how well each alternative satisfies specific criteria. Crucially, they also assign importance to each criterion using the same linguistic scale. The paper then uses a technique inspired by Yager (1981) to derive a single "unit score" for each expert-alternative pair. This involves a min-max operation combined with a negation of the criterion importance, effectively saying that a proposal's score is reduced based on how poorly it performs on important criteria.
  2. Expert Opinion Aggregation: Individual expert evaluations are then combined using Ordered Weighted Averaging (OWA) operators, a technique developed by Yager (1988). This stage involves the decision-maker defining an aggregation function (Q) that specifies how expert opinions should be combined based on the number of experts supporting a proposal. The OWA operator then combines these aggregated expert opinions to produce a final, overall linguistic evaluation for each alternative.

Datasets

Not discussed in the paper. The paper focuses on a methodological framework and illustrates its application with a hypothetical example of project selection.

Key Insights and Ideas

  • Escaping the "Tyranny of Numbers": The most significant insight is the ability to perform complex aggregations using only linguistic scales. This greatly reduces the burden on experts, who can express their judgments more naturally without needing to assign precise numerical values.
  • Approximate Reasoning as a Foundation: The methodology is explicitly rooted in approximate reasoning, formalizing the intuitive way humans often make judgments and decisions.
  • Flexibility in Importance Assignment: The approach allows for importances to be assigned either by the experts themselves or by a central decision-maker, offering flexibility in the process.
  • Linguistic Negation: The paper highlights the importance of a well-defined linguistic negation operation that respects order reversal and involution, crucial for the scoring mechanism.
  • OWA Operators for Consensus: The use of OWA operators provides a structured way to aggregate multiple expert opinions, allowing for different consensus requirements to be modeled.

Key Results

The paper demonstrates the methodology through a detailed example of project selection. It shows how a proposal can be evaluated linguistically (e.g., "medium" overall rating) based on expert inputs and criterion importances. The key result is the successful generation of a comparative linguistic evaluation for multiple projects, enabling the decision-maker to rank and select proposals without relying on numerical scores. The paper emphasizes that this linguistic output, combined with textual evaluations, provides a richer basis for decision-making than purely numerical methods.

Limitations and Future Work

  • Subjectivity of Linguistic Scales: While the paper argues for the intuitive nature of linguistic scales, the interpretation and mapping of these scales can still be subjective. The paper acknowledges that the choice of scale and its interpretation are crucial.
  • Decision-Maker's Role in Defining Q: The effectiveness of the aggregation stage relies on the decision-maker's ability to define an appropriate aggregation function (Q). The paper provides examples of Q but doesn't offer a definitive method for its selection beyond general rational characteristics.
  • Textual Material Integration: While the paper mentions that textual evaluations can supplement the process, it doesn't detail how this textual material is formally integrated into the final decision-making beyond being a qualitative consideration.
  • Future Work: The paper suggests that the textual material provided by experts should be used to provide further distinction amongst projects, implying a need for methods to incorporate this qualitative information more formally. The authors also suggest the process can be used in information fusion, indicating potential for broader applications.

Summary updated

BibTeX
@inproceedings{yager1991non,
  author = {Ronald R. Yager},
  title = {{A Non-Numeric Approach to Multi-Criteria/Multi-Expert Aggregation Based on Approximate Reasoning}},
  booktitle = {Conference on Uncertainty in Artificial Intelligence},
  year = {1991},
}
Figure