An Online Non-Stationary Simulation Optimization Approach Based on Regime Switching

Jianlin Xia, Haowei Wang, Songhao Wang, Szu Hui NgarXiv 2025 · DOI · arXiv

Abstract

Dynamic and evolving operational and economic environments present significant challenges for decision-making. We explore a simulation optimization problem characterized by non-stationary input distributions with regime-switching dynamics across sequential decision stages. This problem encompasses both prediction uncertainty, arising from the regime-switching behavior of input distributions, and input uncertainty, resulting from parameter estimation for these distributions and their dynamics using finite data streams. To address these uncertainties, we develop a Bayesian framework that approximates the true objective function using a Markov Switching Model (MSM). We rigorously validate this approximation by establishing the consistency and asymptotic normality of the objective functions and optimal solutions. To solve the problem in an online fashion, we propose a metamodel-based algorithm that leverages simulation results from previous stages to enhance decision-making. Furthermore, we tackle scenarios with an unknown number of regimes through a Bayesian nonparametric method. Numerical experiments demonstrate that our algorithm achieves superior performance and robust adaptability.

Introduction

In the real world, the "rules of the game" often change abruptly. A supply chain might operate normally for months before a natural disaster or a pandemic shifts it into a "disrupted" regime. Traditional simulation optimization—where computer models are used to find optimal decisions like inventory levels or portfolio weights—usually assumes the environment is static. However, when the environment is non-stationary, decision-makers face two intertwined types of uncertainty: prediction uncertainty regarding future regimes and input uncertainty regarding the statistical parameters of those regimes.

Illustration of regime-switching dynamics where the system transitions between distinct states (Regimes 1, 2, and 3) over time.

This paper introduces Regime-Switching Online Bayesian Simulation Optimization (RSOBSO), a framework designed to bridge this gap by learning and optimizing as new data arrives, accounting for the risk of estimation errors through a Bayesian lens.

Methodology

The RSOBSO framework employs a multi-layered architecture to manage environmental shifts:

  • Markov Switching Model (MSM): The environment is modeled as a hidden Markov chain, allowing the system to calculate transition probabilities between regimes (e.g., the likelihood of moving from a "high growth" state to a "recession").
  • Bayesian Objective Function: Rather than relying on single "plug-in" estimates, the algorithm integrates over the entire posterior distribution of possible parameters to hedge against estimation errors.
  • Unified GP Metamodel: To maintain computational efficiency, the authors utilize a Gaussian Process (GP) that treats both the decision variable ($x$) and the environment parameters ($\lambda$) as inputs. This allows the algorithm to reuse simulation results from previous regimes to inform decisions in new ones.
  • Regime-Aware Expected Improvement (EI): A specialized acquisition function determines the next simulation point by identifying locations that most reduce uncertainty across all likely future regimes.
  • Nonparametric Extension: For scenarios where the number of regimes is unknown, the model integrates a Hierarchical Dirichlet Process (HDP-HMM) to discover new regimes automatically as data evolves.

Experimental Evaluation

The researchers evaluated RSOBSO using synthetic benchmarks and high-stakes real-world scenarios:

  • Synthetic Benchmarks: Tests on 4-regime exponential and 3-regime Gaussian emission problems confirmed the algorithm's ability to track rapid shifts.
  • Inventory Management: Using demand data from the 2008-2009 Great Recession, RSOBSO achieved a significantly lower performance gap compared to traditional benchmark methods.
  • Portfolio Optimization: In a two-asset allocation task covering the financial crisis, RSOBSO outperformed benchmarks by quickly identifying the market recovery in May 2009 and adjusting allocations more aggressively than static models.

Key Insights and Results

A primary innovation of this work is the joint modeling of decision variables and environmental parameters within the GP metamodel. This allows the model to understand why outputs change (due to shifts in $\lambda$), enabling continuous learning across regime boundaries. Furthermore, the authors provide theoretical proof that the estimated optimal solution converges to the true optimum at a rate of $O_p(1/\sqrt{t})$, establishing a rigorous statistical foundation for the approach.

While the current model assumes discrete regimes and faces scaling challenges common to Gaussian Processes ($O(n^3)$), it demonstrates superior downside protection and robustness to unknown environmental structures compared to non-regime-aware models.

Conclusion

RSOBSO offers a mathematically rigorous approach to optimization in volatile environments. By combining Markov Switching Models with Bayesian Gaussian Processes, it provides a robust mechanism for decision-making that accounts for both the uncertainty of future regime shifts and the noise inherent in real-time data streams.

Summary updated

Abstract

Dynamic and evolving operational and economic environments present significant challenges for decision-making. We explore a simulation optimization problem characterized by non-stationary input distributions with regime-switching dynamics across sequential decision stages. This problem encompasses both prediction uncertainty, arising from the regime-switching behavior of input distributions, and input uncertainty, resulting from parameter estimation for these distributions and their dynamics using finite data streams. To address these uncertainties, we develop a Bayesian framework that approximates the true objective function using a Markov Switching Model (MSM). We rigorously validate this approximation by establishing the consistency and asymptotic normality of the objective functions and optimal solutions. To solve the problem in an online fashion, we propose a metamodel-based algorithm that leverages simulation results from previous stages to enhance decision-making. Furthermore, we tackle scenarios with an unknown number of regimes through a Bayesian nonparametric method. Numerical experiments demonstrate that our algorithm achieves superior performance and robust adaptability.
BibTeX
@misc{xia2025online,
  author = {Jianlin Xia and Haowei Wang and Songhao Wang and Szu Hui Ng},
  title = {{An Online Non-Stationary Simulation Optimization Approach Based on Regime Switching}},
  year = {2025},
  eprint = {2508.12634},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2508.12634},
  howpublished = {arXiv},
}
Figure