SDG and qualitative trend based model multiple scale validation

Dong Gao, Xin Xu, Jianjin Yin, Hongyu Zhang, Beike ZhangIOP 2017 · DOI

Abstract

Verification, Validation, and Accreditation (VV&A) are essential technologies in simulation and modeling. Traditional validation methods often suffer from weak completeness, reliance on a single scale of analysis, and heavy dependence on subjective human experience. This paper proposes a novel multi-scale validation framework based on Signed Directed Graphs (SDG) and qualitative trend analysis. By building an SDG model and incorporating qualitative trends, complete testing scenarios are generated through positive inference. Validation is then performed by comparing these scenarios with simulation outputs across different scales of similarity. The effectiveness of this approach is demonstrated through a case study involving a reactor simulation model.

Introduction

In complex system engineering, simulation models are indispensable for design and testing. However, the utility of these models depends entirely on their accuracy, which is ensured through VV&A. Traditional validation faces significant hurdles: the scarcity of real-world failure data, the subjectivity of expert-led assessments, and "scale rigidity"—the inability to capture nuanced changes like acceleration or curvature over time. This research introduces a more objective, multi-scale validation process that leverages qualitative reasoning to overcome these limitations.

Methodology

The proposed framework integrates structural modeling with qualitative trend analysis to create a robust validation pipeline.

SDG Modeling and Trend Primitives

The process begins with a Signed Directed Graph (SDG), a qualitative causal model where nodes represent system variables and edges represent their relationships. To move beyond simple binary states, the authors utilize six basic qualitative trend primitives (A through F). These primitives represent unique combinations of slope (first derivative) and curvature (second derivative), allowing the model to capture the "shape" of a process.

Qualitative trend primitives representing different combinations of slope and curvature.

Positive Inference and Scenario Generation

By applying inference rules to the SDG, the system automatically generates comprehensive testing scenarios. This "Positive Inference" step determines how every connected variable must respond if a specific node begins to exhibit a certain trend. This allows for the testing of rare failure modes that may not exist in historical datasets.

Multi-Scale Similarity Validation

Validation is assessed across three hierarchical levels:

  • Level 1: State matching (e.g., threshold crossings).
  • Level 2: Directional matching (first derivative).
  • Level 3: Curvature matching (second derivative).

A total similarity weight ($S_{total}$) is calculated to assign a final validation grade, providing a granular view of model accuracy.

The complete workflow for SDG and qualitative trend-based multi-scale validation.

Case Study and Results

The methodology was applied to the SMPT1000 reactor simulation model, tracking 22 main variables including flow rates, levels, temperatures, and pressures. Using the SDG, the researchers generated 40 primary validation scenarios and 120 detailed trend scenarios.

In a specific test case involving the cooling water feed flow (FI1201), the calculated similarity weight was 2.18. According to the defined similarity levels, any score exceeding 2.0 qualifies as Level 3 Similarity. This result confirms that the simulation model accurately reflects not only the states and directions of the physical system but also the nuances of its rate of change.

Discussion and Conclusion

By using qualitative trends as "quantitative-lite" data, this framework captures essential system dynamics without the noise of exact numerical values. The automated generation of scenarios significantly improves testing completeness. However, the method's success depends on the accuracy of the initial SDG. Future work will address handling undetermined states ("?") in complex systems and scaling the traversal algorithms for massive industrial plants with thousands of variables.

Summary updated

Abstract

Verification, Validation and Accreditation (VV&A) is key technology of simulation and modelling. For the traditional model validation methods, the completeness is weak; it is carried out in one scale; it depends on human experience. The SDG (Signed Directed Graph) and qualitative trend based multiple scale validation is proposed. First the SDG model is built and qualitative trends are added to the model. And then complete testing scenarios are produced by positive inference. The multiple scale validation is carried out by comparing the testing scenarios with outputs of simulation model in different scales. Finally, the effectiveness is proved by carrying out validation for a reactor model.
BibTeX
@inproceedings{gao2017sdg,
  author = {Dong Gao and Xin Xu and Jianjin Yin and Hongyu Zhang and Beike Zhang},
  title = {{SDG and qualitative trend based model multiple scale validation}},
  booktitle = {IOP},
  year = {2017},
  doi = {10.1088/1757-899x/231/1/012033},
}
Figure