A decision-making process for exploring architectural variants in systems engineering

Jérôme Le Noir, Sébastien Madelénat, Grégory Gailliard, Christophe Labreuche, Mathieu Acher, Olivier Barais, Olivier ConstantSPLC 2016 · DOI

TL;DR

This paper presents a novel, tool-supported process for systematically exploring and selecting the best architectural variants in complex systems engineering projects. By combining variability modeling with multi-criteria decision analysis, the approach aims to automate the often manual and subjective process of architectural decision-making, leading to more rational and justifiable choices.

What Problem Does This Paper Address?

In systems engineering, choosing the optimal architectural variant from a multitude of possibilities is a critical but challenging task. This process is typically manual, time-consuming, and prone to errors and subjectivity. Engineers often rely on ad-hoc methods for exploring and justifying architectural solutions, leading to a lack of transparency and difficulty in explaining the rationale behind decisions. The paper addresses this gap by proposing a structured, model-driven approach that integrates architectural design with decision-making frameworks to automate and rationalize this crucial phase.

Methods

The paper introduces a two-pronged approach:

  1. Modeling and Deriving Architectural Variants:
    • Common Variability Language (CVL): Used to formally represent architectural variability and constraints, defining the "problem space" of possible configurations.
  2. Feature Models: Employed to capture high-level design choices and their relationships.
  3. Variability Realization Model (VRM): Links feature selections to specific model elements, defining how to modify or combine base models.
  4. Custom CVL Derivation Engine: Automates the generation of concrete architectural variants based on selected configurations. This engine supports various semantics for variation points, including existence, substitution, and value assignment.
  5. Pattern Technology (EMF Diff/Merge Patterns): Used to weave model components and instantiate them into derived models, facilitating the creation of reusable architectural building blocks.
  6. Decision-Making Process (MYRIAD):
    • Preference Model: A structured representation of stakeholder preferences, built through a multi-stage process:
  7. Structuring Phase: Identifying and organizing metrics and criteria into a hierarchical tree.
  8. Criteria Construction: Defining utility functions for each metric, mapping values to satisfaction degrees (0-1). This involves identifying reference levels like "Completely Satisfactory," "Budget," and "Not Satisfied at All." * Aggregation Construction: Defining how criteria are combined using aggregation functions. The paper highlights the use of the 2-additive Choquet integral, which can capture complex interactions like veto, favor, and complementarity between criteria, going beyond simple weighted sums.
  9. Assessment and Decision Support: The preference model is applied to the derived architectural variants to compute an overall evaluation score. The system provides graphical and textual explanations to justify the scores and facilitate decision-making.

Datasets

Not discussed in the paper. The paper focuses on an industrial case study involving a secure radio platform (SDR) and does not present a formal dataset for training or evaluation.

Key Insights and Ideas

  • Bridging the Gap: The core innovation is the seamless integration of model-based systems engineering (MBSE) tools for variant generation with multi-criteria decision analysis (MCDA) tools for evaluation. This addresses the common disconnect between design and decision-making.
  • Beyond Weighted Sums: The explicit use of the 2-additive Choquet integral is crucial. It allows for the representation of nuanced interactions between criteria (e.g., one criterion being a "veto" for another), which is often necessary for complex real-world systems and leads to more realistic evaluations than simple weighted sums.
  • Justifiable Decisions: The approach emphasizes generating not just scores but also explanations for the evaluations. This "argumentation report" is vital for stakeholders to understand why a particular variant is preferred, increasing confidence and transparency.
  • Automation of a Manual Process: The paper demonstrates how a significant portion of the laborious and subjective architectural exploration and comparison process can be automated, freeing up engineers to focus on higher-level design challenges.
  • Tool Integration is Key (and a Challenge): While the approach is powerful, the reliance on integrating multiple specialized tools (pure::variants, KCVL, FAMILIAR, Capella, MYRIAD) presents a significant effort for system architects to master.

Key Results

The paper reports on an industrial case study for a secure radio platform (SDR).

  • Variant Reduction: The initial feature model allowed for 144 potential variants, which were constrained to five (S1-S5) for the study.
  • Evaluation Outcomes: At 1000 MHz, no solution was deemed fully satisfying, with power consumption being a major issue. At 800 MHz, only variants S4 and S5 were found to be operationally acceptable.
  • Variant S4 Analysis: Detailed analysis of S4 showed strong performance in security and safety (SW partitioning) but weakness in hardware partitioning. The complementarity between "SoC Lifetime" and "CPU Resource" was highlighted, with "SoC Lifetime" acting as a veto criterion.
  • Improved Decision Confidence: The tooled process led to better confidence in decisions by bringing objectivity and enabling justification.
  • Realistic Preference Modeling: The use of the Choquet integral correctly identified S4 as the best variant, unlike a simple weighted sum which favored S3, aligning with the architect's intuition due to the complex interactions between criteria.

Limitations and Future Work

  • Preference Model Construction: The mapping of criteria into aggregations can be questionable and requires expert consensus. The method currently struggles with a very high number of criteria or when preference sorting is not straightforward.
  • Manual Binding: The binding of the preference model to the derived architectural variants is currently done manually, limiting full automation. The authors suggest this binding should be integrated into the variability model.
  • Tooling Integration Effort: The current methodology relies on integrating six different tools, which requires significant effort for system architects to master.
  • Future Work: The authors suggest further effort is needed to improve tooling integration and explore the applicability of the approach in other application domains. They also note the need to address the manual binding of preference models to architectural variants.

Summary updated

BibTeX
@inproceedings{noir2016decision,
  author = {Jérôme Le Noir and Sébastien Madelénat and Grégory Gailliard and Christophe Labreuche and Mathieu Acher and Olivier Barais and Olivier Constant},
  title = {{A decision-making process for exploring architectural variants in systems engineering}},
  booktitle = {SPLC},
  year = {2016},
  doi = {10.1145/2934466.2946045},
}

Loading related content...

Figure