Abstract
Approximate Bayesian Computation (ABC) are likelihood-free Monte Carlo methods. ABC methods use a comparison between simulated data, using different parameters drew from a prior distribution, and observed data. This comparison process is based on computing a distance between the summary statistics from the simulated data and the observed data. For complex models, it is usually difficult to define a methodology for choosing or constructing the summary statistics. Recently, a nonparametric ABC has been proposed, that uses a dissimilarity measure between discrete distributions based on empirical kernel embeddings as an alternative for summary statistics. The nonparametric ABC outperforms other methods including ABC, kernel ABC or synthetic likelihood ABC. However, it assumes that the probability distributions are discrete, and it is not robust when dealing with few observations. In this paper, we propose to apply kernel embeddings using an smoother density estimator or Parzen estimator for comparing the empirical data distributions, and computing the ABC posterior. Synthetic data and real data were used to test the Bayesian inference of our method. We compare our method with respect to state-of-the-art methods, and demonstrate that our method is a robust estimator of the posterior distribution in terms of the number of observations.
Introduction
In many scientific domains, researchers encounter complex stochastic models where the likelihood function—the probability of observing data given a set of parameters—is computationally intractable. To perform inference in these scenarios, Approximate Bayesian Computation (ABC) has become a standard tool. The traditional ABC approach relies on comparing simulated data to observed data via manually selected "summary statistics" (e.g., mean, variance).
However, selecting these statistics is often a significant challenge. Poorly chosen statistics can lead to a loss of information, while too many statistics can make the algorithm computationally inefficient. While recent nonparametric methods have attempted to use kernel embeddings to compare distributions directly, these often struggle with small sample sizes or assume discrete data distributions. This paper introduces PABC (Parzen-based ABC), which utilizes a smoother density estimator to provide a more robust distance metric for likelihood-free inference.
Methodology
The core innovation of PABC is the replacement of discrete empirical distributions with a Parzen-window density estimator. By smoothing the data distributions before mapping them into a Reproducing Kernel Hilbert Space (RKHS), the method creates a more stable distance metric.
The Parzen Metric
The authors derive a new distance metric, $\gamma_k^{Parzen}$, which assumes the underlying density is continuous. While this smoothing introduces a controlled amount of bias, it significantly reduces the variance of the estimator. This trade-off is particularly beneficial when the number of observations is limited.
Sequential Monte Carlo (SMC) Integration
PABC is designed to be compatible with Sequential Monte Carlo (SMC) frameworks. In an ABC-SMC setup, parameter estimates are refined through successive importance sampling steps, using the Parzen-based metric to evaluate the "closeness" of simulated data at each stage.
Experimental Evaluation
The proposed method was evaluated against two benchmarks:
- Uniform Mixture Model: A synthetic toy problem where the true posterior is known, allowing for precise calculation of the Root Mean Squared Error (RMSE).
- Blowfly Population Dynamics: A complex, nonlinear ecological system based on real-world data, known for its transitions between stability and chaos.
Results and Discussion
PABC demonstrates significant improvements in robustness and accuracy over standard ABC and the previous state-of-the-art kernel method, K2ABC.
Robustness to Sample Size
One of the primary advantages of PABC is its stability across different dataset sizes. As shown in the figure below, PABC maintains the lowest and most consistent RMSE as the number of observations increases from 40 to 400. Specifically, PABC achieved an RMSE of $0.0696 \pm 0.0006$, significantly outperforming the more volatile ABC and K2ABC methods.
Posterior Estimation Accuracy
In the toy mixture model, standard ABC methods often failed to correctly estimate specific mixing coefficients (notably $\theta_3$ and $\theta_5$). As illustrated in the comparison of posterior means, PABC and its SMC variant provide estimates that are much closer to the true underlying values, effectively capturing the "shape" of the distribution without requiring expert-defined summary statistics.
Furthermore, in the blowfly population study, PABC achieved a median cross-correlation of $\rho = 0.6501$ between simulated and observed time series, surpassing both K2ABC and traditional summary-statistic-based ABC approaches.
Conclusion
PABC offers a robust, automated alternative to the "summary statistic headache" in Approximate Bayesian Computation. By leveraging Parzen-window smoothing within a kernel embedding framework, the method provides superior stability and accuracy, especially for small datasets. Future research will focus on optimizing kernel bandwidth selection and exploring the application of PABC to other complex domains, such as electrical network analysis.
Summary updated