Abstract Simulation model validation and calibration (SMVC) is a complex endeavor involving uncertainty description, extensive simulation execution, and intricate data analysis. Precise modeling requires accounting for various uncertainty factors, such as model form assumptions, solution approximations, and random variability in inputs. This paper presents a software platform developed to assist in the SMVC process under conditions of uncertainty. We introduce an unprecedented process model encompassing uncertainty description, simulation experiment design, model validation, and model calibration. The platform's design is bifurcated into structural and functional components, utilizing the "strategy" design pattern to integrate and maintain diverse SMVC algorithms. The platform's utility is demonstrated through a flight vehicle kinematic control system case study, showing significant improvements in model consistency after calibration.
Introduction
In modern engineering, high-fidelity simulation models are indispensable for predicting system behavior in fields like aerospace and power generation. However, these models are inherently limited by two types of uncertainty: aleatory (stochastic randomness) and epistemic (lack of knowledge). Quantifying these uncertainties and aligning model outputs with real-world observations—the core of SMVC—is traditionally a fragmented and tedious process. Engineers often struggle with disparate tools for data management, statistical analysis, and optimization. There is a critical need for a unified environment that manages the entire lifecycle of simulation credibility.
The HIT-MVCP Process Model
To address these challenges, researchers at the Harbin Institute of Technology developed HIT-MVCP, an integrated platform built upon a comprehensive six-step workflow. This process model ensures that uncertainty is accounted for at every stage of the simulation lifecycle.
As illustrated in the process model, the workflow begins with Uncertainty Description (A1) and Experiment Schemes Design (A2). These feed into Simulation Execution (A3). The resulting model outputs are compared against experimental observations during Simulation Model Validation (A4). Finally, the calibration phase utilizes Consistency Metamodeling (A5) and Uncertainty Parameters Optimization (A6) to refine the model.
Platform Architecture and Design
The HIT-MVCP platform is designed with modularity and extensibility as core requirements. The architecture is divided into several functional subsystems, coordinated through a central data management layer.
Key Design Pillars:
- Modular Subsystems: The platform separates concerns into Uncertainty, Experiment, Validation, and Calibration subsystems. This mirrors the actual engineering workflow and allows for independent updates to specific modules.
- The Strategy Design Pattern: To prevent the platform from becoming obsolete, the C++ implementation utilizes the "strategy pattern." This allows users to swap algorithms (e.g., changing sampling methods or optimization routines) without altering the core software architecture.
- Evidence-Based Validation: The platform employs an "evidence distance" approach for validation. This method is particularly effective for multi-output systems, as it converts complex data features into mathematical "evidence bodies" for robust comparison.
- Metamodeling for Efficiency: To reduce the computational burden of running high-fidelity simulations repeatedly during calibration, the platform constructs metamodels—efficient mathematical approximations used to find optimal parameters.
Case Study: Flight Vehicle Kinematic Control
The platform was evaluated using a flight vehicle kinematic control system simulation. The model accounted for aleatory uncertainties (atmospheric density, initial velocity) and epistemic uncertainties (initial mass, reference area).
Initially, the model showed a consistency score of 0.6581 when compared to experimental data. By utilizing the HIT-MVCP calibration loop, the epistemic parameters were optimized, resulting in a significantly improved validation score of 0.9398. This demonstrates the platform's ability to effectively "close the loop" between validation results and model refinement.
Conclusion and Future Work
HIT-MVCP represents a significant advancement in streamlining the SMVC process. By integrating validation results directly into the calibration objective function, the platform transforms validation from a passive assessment into an active driver of model accuracy. Future developments will focus on incorporating more sophisticated uncertainty theories, such as Evidence Theory and Imprecise Probability, and expanding the algorithm library to support increasingly complex, non-linear simulation models.
Summary updated