A New Stochastic Approximation Method for Gradient-based Simulated Parameter Estimation

Zehao Li, Yijie PengarXiv 2025 · OpenReview

Abstract

This paper tackles the challenge of parameter calibration in stochastic models, particularly in scenarios where the likelihood function is unavailable in an analytical form. We introduce a gradient-based simulated parameter estimation framework, which employs a multi-time scale stochastic approximation algorithm. This approach effectively addresses the ratio bias that arises in both maximum likelihood estimation and posterior density estimation problems. The proposed algorithm enhances estimation accuracy and significantly reduces computational costs, as demonstrated through extensive numerical experiments. Our work extends the GSPE framework to handle complex models such as hidden Markov models and variational inference-based problems, offering a robust solution for parameter estimation in challenging stochastic environments.

Introduction

In fields ranging from financial risk assessment to medical diagnostics, researchers rely on stochastic simulators to model complex systems. These "black-box" simulators often lack an analytical likelihood function—the probability of observing data given specific parameters. Calibrating these models typically requires the gradient of the log-likelihood, which is mathematically expressed as a ratio: the gradient of the density divided by the density itself.

A fundamental challenge in this context is "ratio bias." While Monte Carlo simulations can provide unbiased estimates of the numerator and denominator independently, the ratio of two unbiased estimators is almost always biased. Traditional methods attempt to mitigate this by using massive sample sizes per iteration to make the bias negligible, which is computationally prohibitive. This research introduces a framework to eliminate this bias asymptotically without requiring enormous batches of simulation data.

Methodology

The core of the proposed Gradient-based Simulated Parameter Estimation (GSPE) framework is a Multi-Time Scale (MTS) stochastic approximation approach. Instead of directly calculating noisy ratios at each step, the problem is treated as a coupled root-finding exercise involving two distinct scales:

  1. The Faster Scale ($D_k$): This process tracks and "learns" the gradient of the log-likelihood.
  2. The Slower Scale ($\theta_k$): This process updates the actual model parameters based on the current state of the faster scale.

By ensuring the gradient-tracking moves faster than the parameter-updating (specifically, $\beta_k / \alpha_k \to 0$), the bias in the gradient estimator effectively "averages out" over time. This decoupling allows the algorithm to converge to the true parameters without the instability inherent in direct division by noisy estimators.

Algorithm 1: Multi-Time Scale Stochastic Approximation for GSPE.

The framework is extended to three specific scenarios:

  • Maximum Likelihood Estimation (MLE): Identifying the optimal parameter set.
  • Posterior Density Estimation (PDE): Utilizing Variational Inference within a Nested MTS algorithm to find a distribution of parameters.
  • Hidden Markov Models (HMM): Integrating the framework with Sequential Monte Carlo (Particle Filtering) to handle latent states.

Experimental Evaluation

The authors evaluated the MTS algorithm using several controlled stochastic simulations:

  • Linear Stochastic Model: $Y = X_1 + \theta X_2$ (with Gaussian $X$ variables), providing a ground truth analytical MLE for comparison.
  • Gaussian PDE Model: A latent variable model with a standard normal prior used to test Variational Inference (ELBO) optimization.
  • HMM State-Space Model: A model with evolving hidden states used to test integration with particle filters.

Key Results and Discussion

The MTS approach demonstrates significant advantages over traditional Single-Time Scale (STS) methods:

  • Improved Accuracy: In linear model tests, the MTS algorithm achieved an absolute bias of $5.9 \times 10^{-5}$ with a budget of $10^8$ samples, whereas the STS method remained at $1 \times 10^{-3}$. In HMM applications with 1,000 particles, MTS reduced the Mean Absolute Error from 0.0145 to 0.0104.
  • Computational Efficiency: While STS methods often require $10^5$ or more samples per iteration to suppress bias, the MTS approach achieves superior accuracy with significantly fewer samples (e.g., $10^3$ or $10^4$), making it practical for complex simulators.
  • Numerical Stability: By avoiding direct division by potentially small, noisy denominator estimators, the MTS algorithm exhibits smoother convergence trajectories and greater reliability compared to the erratic behavior of "plug-in" estimators.

Conclusion

This paper presents a robust solution for parameter estimation in stochastic models where likelihoods are intractable. By leveraging multi-time scale stochastic approximation, the GSPE framework effectively eliminates ratio bias and improves computational efficiency. Future work will focus on scaling these updates to high-dimensional parameter spaces, such as those found in deep neural networks, and further optimizing step-size selection. While theoretical results for the HMM case are currently being finalized, the empirical evidence strongly supports the MTS approach as a superior alternative for complex simulator calibration.

Summary updated

Abstract

This paper tackles the challenge of parameter calibration in stochastic models, particularly in scenarios where the likelihood function is unavailable in an analytical form. We introduce a gradient-based simulated parameter estimation framework, which employs a multi-time scale stochastic approximation algorithm. This approach effectively addresses the ratio bias that arises in both maximum likelihood estimation and posterior density estimation problems. The proposed algorithm enhances estimation accuracy and significantly reduces computational costs, as demonstrated through extensive numerical experiments. Our work extends the GSPE framework to handle complex models such as hidden Markov models and variational inference-based problems, offering a robust solution for parameter estimation in challenging stochastic environments.
BibTeX
@misc{li2025new,
  author = {Zehao Li and Yijie Peng},
  title = {{A New Stochastic Approximation Method for Gradient-based Simulated Parameter Estimation}},
  year = {2025},
  eprint = {2503.18319},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2503.18319},
}
Figure