Abstract: A lower bound is an important tool for predicting the performance that an estimator can achieve under a particular statistical model. Bayesian bounds are a kind of such bounds which not only utilizes the observation statistics but also includes the prior model information. In reality, however, the true model generating the data is either unknown or simplified when deriving estimators, which motivates the works to derive estimation bounds under modeling mismatch situations. This paper provides a derivation of a Bayesian Cramér-Rao bound under model misspecification, defining important concepts such as pseudotrue parameter that were not clearly identified in previous works. The general result is particularized in linear and Gaussian problems, where closed-forms are available and results are used to validate the results.
Introduction
In signal processing and machine learning, estimators are typically designed based on an assumed mathematical model of the data-generating process. However, as the adage goes, "all models are wrong." In real-world applications, the true data-generating process is often more complex or slightly different from the assumed model—a phenomenon known as model misspecification.
Standard performance benchmarks, such as the Bayesian Cramér-Rao Bound (BCRB), assume perfect model knowledge. Under mismatch, these standard bounds can become loose or entirely invalid, leading to over-optimistic predictions of how well an algorithm will perform. This paper introduces the Misspecified Bayesian Cramér-Rao Bound (MBCRB) to provide a rigorous theoretical framework for predicting performance limits in these mismatched scenarios.
Theoretical Framework
The authors bridge the gap in existing literature by deriving a parametric MBCRB. Their approach hinges on several key technical steps:
- The Pseudotrue Parameter ($\theta_0$): They define a mapping from the true parameter space ($\psi$) to the assumed parameter space ($\theta$). This "pseudotrue" value minimizes the Kullback-Leibler (KL) divergence between the true distribution and the assumed joint distribution.
- Parametric Transformation: Instead of treating mismatch as an intractable error, they treat it as a transformation, mapping the BCRB of the true parameters through the geometry of the misspecified model.
- The "Sandwich" Bound: The resulting bound takes a "sandwich" form, involving the inverse of the Bayesian Fisher Information Matrix (BFIM) of the true model surrounded by the expectations of the Jacobian of the pseudotrue parameter mapping.
Numerical Validation
The proposed framework was validated using synthetic Monte Carlo simulations of a linear Gaussian system. The study utilized 100,000 independent realizations per data point to ensure statistical significance across various mismatch scenarios, including errors in prior means, observation scaling, and noise covariance structures.
As shown in Figure 1, the MBCRB tracks the actual Root Mean Square Error (RMSE) of a Maximum A Posteriori (MAP) estimator almost perfectly when evaluating the error relative to the pseudotrue parameter. This demonstrates the extreme tightness of the bound across different sample sizes.
When considering total estimation error, the standard BCRB often fails to accurately characterize performance. Figure 2 illustrates that while the standard BCRB can become invalid under mismatch, the MBCRB provides a reliable lower bound for the estimator's variance. The total error can be decomposed into the MBCRB (representing variance) plus a "bias" term representing the distance between the true parameter and the pseudotrue value.
Conclusion
The MBCRB provides a necessary generalization of Bayesian performance bounds for real-world systems where model mismatch is inevitable. By defining the pseudotrue realization, the authors have created a tool that accurately predicts how performance degrades as the level of mismatch increases. This framework is particularly useful for validating estimators like the Quasi-Maximum Likelihood Estimator (QMLE) in complex environments.
Summary updated