Using simulation to incorporate dynamic criteria into multiple criteria decision-making

Uwe Aickelin, Jenna Marie Reps, Peer-Olaf Siebers, Peng LiJournal of the Operational Research Society 2017 · DOI · OpenReview

TL;DR

This paper introduces a novel approach to multi-criteria decision-making (MCDA) by integrating discrete event simulation (DES) with traditional MCDA methods. This allows for the incorporation of dynamic and uncertain criteria, which are often overlooked in standard MCDA and cost-benefit analysis (CBA), leading to potentially more robust long-term decisions.

What Problem Does This Paper Address?

Making complex decisions often involves evaluating multiple criteria, some of which are not easily quantifiable or are subject to change over time. Traditional methods like Cost-Benefit Analysis (CBA) are limited to static monetary values, while standard Multi-Criteria Decision Analysis (MCDA) typically handles static criteria, whether monetary or non-monetary. This leaves a significant gap when dealing with decisions where future outcomes are uncertain or evolve dynamically, such as infrastructure planning or resource management. The paper addresses this by proposing a method to incorporate these dynamic and uncertain criteria into the decision-making process, aiming to provide a more realistic and comprehensive evaluation of options.

Methods

The core of the paper's methodology is the combination of Discrete Event Simulation (DES) and Multi-Criteria Decision Analysis (MCDA).

  • Discrete Event Simulation (DES): This technique is used to model and simulate the behavior of a system over time, focusing on discrete events. In this study, DES is employed to generate performance measures for criteria that are dynamic and uncertain. By simulating various scenarios, the method can estimate how different decision options would perform under a range of future conditions.
  • Multi-Criteria Decision Analysis (MCDA): This is a standard decision-aiding technique used to evaluate options based on multiple, often conflicting, criteria. The paper uses MCDA to aggregate the scores of each option across all criteria (both static and those informed by simulation) to arrive at a final ranking.
  • Combined Approach: The novel aspect is the integration: DES is run first to generate data for dynamic and uncertain criteria. These simulated results are then fed into the MCDA framework alongside static criteria. This allows for a more holistic evaluation, especially for decisions requiring long-term foresight. The paper also discusses sensitivity analysis to assess the robustness of the results.

Datasets

The paper uses a case study of the Port of Dover for its evaluation.

  • Port of Dover Data: This includes historical financial reports (2010-2011) for cost estimations, traffic volume data, and discussions with Dover Harbour Board officials to define scenarios and criteria.
  • Simulated Scenarios: The paper defines nine future scenarios based on varying annual vehicle traffic growth (VTG) and the percentage of lorries (LTP). These scenarios are crucial inputs for the DES.
  • No new benchmark datasets were introduced. The focus is on applying the methodology to a real-world problem.

Key Insights and Ideas

  • Bridging the Gap: The most significant insight is the successful integration of simulation with MCDA to handle dynamic and uncertain criteria, a limitation of traditional methods.
  • Long-Term Decision Making: The approach is particularly valuable for decisions with long-term implications, where uncertainty is inherent and short-sightedness can be detrimental.
  • Divergent Results: The dynamic MCDA approach can lead to different optimal solutions compared to static MCDA or CBA, highlighting the importance of considering dynamic factors.
  • Complexity vs. Benefit: While simulation adds complexity, it can provide crucial insights that might be missed otherwise, making it a beneficial tool when uncertainty is high.

Key Results

The case study at the Port of Dover aimed to identify the optimal infrastructure expansion strategy. The study compared three options:

  1. Do nothing.
  2. Expand the road network by adding a lorry lane at the weighbridge.
  3. Expand the road network by adding a non-lorry lane at the weighbridge.

The dynamic MCDA, incorporating simulation for customer satisfaction (queue frequency, passive queuing, and dissatisfaction), identified Option 2 (adding a lorry lane) as the optimal solution. This differed from static MCDA or CBA, which would have favored Option 1 (doing nothing). Sensitivity analysis indicated that Option 2 remained the most stable optimal choice under various weighting and score perturbations.

Limitations and Future Work

  • Assumptions and Complexity: The authors acknowledge that the MCDA construction involved numerous assumptions, and the simulation adds complexity to the decision-making process.
  • Weighting Sensitivity: While sensitivity analysis was performed, the final decision can still be influenced by the subjective weighting of criteria. The paper notes that if cost is weighted significantly higher, Option 1 could become optimal.
  • Future Work: The authors suggest validating the approach across more scenarios and different simulations. They also propose making the simulation objects "more active," implying a potential for more sophisticated modeling of dynamic interactions. Further research could explore applying this method to other domains with similar dynamic and uncertain decision contexts.

Summary updated

Abstract

In this paper, we present a case study demonstrating how dynamic and uncertain criteria can be incorporated into a multicriteria analysis with the help of discrete event simulation. The simulation guided multicriteria analysis can include both monetary and non-monetary criteria that are static or dynamic, whereas standard multi criteria analysis only deals with static criteria and cost benefit analysis only deals with static monetary criteria. The dynamic and uncertain criteria are incorporated by using simulation to explore how the decision options perform. The results of the simulation are then fed into the multicriteria analysis. By enabling the incorporation of dynamic and uncertain criteria, the dynamic multiple criteria analysis was able to take a unique perspective of the problem. The highest ranked option returned by the dynamic multicriteria analysis differed from the other decision aid techniques. The results suggest that dynamic multiple criteria analysis may be highly suitable for decisions that require long-term evaluation, as this is often when uncertainty is introduced.
BibTeX
@article{aickelin2017using,
  author = {Uwe Aickelin and Jenna Marie Reps and Peer-Olaf Siebers and Peng Li},
  title = {{Using simulation to incorporate dynamic criteria into multiple criteria decision-making}},
  journal = {Journal of the Operational Research Society},
  year = {2017},
  doi = {10.1080/01605682.2017.1410010},
}
Figure