Abstract
While simulations have been utilized in diverse domains, such as urban growth modeling, market dynamics modeling, etc; some of these applications may require validations based upon some real-world observations modeled in the simulation, as well. This validation has been categorized into either qualitative face-validation or quantitative empirical validation, but as the importance and the accumulation of data grows, the importance of the quantitative validation has been highlighted in the recent studies, i.e. digital twin. The key component of quantitative validation is finding a calibrated set of parameters to regenerate the real-world observations with simulation models. While this parameter calibration has been fixed throughout a simulation execution, this paper expands the static parameter calibration in two dimensions: dynamic calibration and heterogeneous calibration. First, dynamic calibration changes the parameter values over the simulation period by reflecting the simulation output trend. Second, heterogeneous calibration changes the parameter values per simulated entity clusters by considering the similarities of entity states. We experimented the suggested calibrations on one hypothetical case and another real-world case. As a hypothetical scenario, we use the Wealth Distribution Model to illustrate how our calibration works. As a real-world scenario, we selected Real Estate Market Model because of three reasons. First, the models have heterogeneous entities as being agent-based models; second, they are economic models with real-world trends over time; and third, they are applicable to the real-world scenarios where we can gather validation data.
Introduction
Agent-Based Models (ABMs) are powerful tools for simulating complex systems like urban growth or stock markets. However, they often suffer from "simulation divergence," where model outputs naturally drift away from real-world observations over time. Traditionally, researchers calibrate these models manually or use static parameters that remain uniform throughout the simulation run and across all agents. This approach fails to account for two critical realities: economic or social regimes shift over time (e.g., market booms vs. crashes), and agents within a system are inherently diverse with varying motivations.
This paper presents an automated machine learning framework designed to calibrate ABMs across two new dimensions: time (dynamic calibration) and agent diversity (heterogeneous calibration). By automating the determination of when to change parameters and how to differentiate them for various agent groups, the framework significantly reduces simulation error compared to traditional methods.
Methodology
The proposed framework introduces an iterative Calibration Framework Cycle that alternates between two primary components to handle temporal and agent-level complexity.
Dynamic and Heterogeneous Calibration
Dynamic Calibration addresses temporal changes by utilizing a variant of the Hidden Markov Model (HMM). The framework analyzes deviations between simulation results and real-world data to identify "hidden regimes"—specific time periods where the model's behavior must shift. Parameters are then updated specifically for these identified blocks, ensuring the model tracks evolving trends without the computational burden of per-timestep calibration.
Heterogeneous Calibration manages agent diversity. The framework first employs a Variational Autoencoder (VAE) to compress complex agent state data into a lower-dimensional latent space. It then applies clustering (such as GMM or DPMM) to group similar agents. Finally, Gaussian Process-based Bayesian Optimization is used to identify optimal parameters for each specific cluster.
As shown in the comparative analysis, the proposed framework offers unique advantages in calibrating dynamic and heterogeneous parameters where traditional derivative-based or heuristic algorithms may struggle with global optimality or high dimensionality.
Experimental Evaluation
The framework was evaluated using two distinct scenarios:
- Wealth Distribution Model (Synthetic): Based on the "Sugarscape" model, this scenario used a synthesized validation dataset with known ground-truth parameters to verify the framework's recovery capabilities.
- Real Estate Market Model (Real-World): A complex model of the South Korean housing market using macro-level data from the Korea Appraisal Board (KAB), including sales/lease price indices and transaction volumes.
Key Results
The automated framework consistently outperformed both random search and manual human calibration baselines. In the real estate market case, the "Human Calibration" baseline resulted in a Total Mean Absolute Percentage Error (MAPE) of 0.214. The proposed framework reduced this error to 0.105, representing a nearly 50% improvement in accuracy. Furthermore, the framework achieved superior accuracy in fewer than 20 iterations, whereas a standard random search required over 80 iterations just to reach the human baseline. Qualitative visualizations confirmed that the calibrated model closely followed the volatile transaction peaks and valleys present in the real-world KAB data.
Conclusion
This research represents a significant step toward robust "Digital Twins," where simulations adapt dynamically as real-world data flows in. While the framework demonstrates high performance, future work will focus on reducing the computational cost of running complex ABMs within the Bayesian Optimization loop and exploring the deeper interactions between dynamic and heterogeneous parameter dimensions.
Summary updated