Active Recursive Bayesian Inference with Posterior Trajectory Analysis Using α-Divergence

Yeganeh M. Marghi, Aziz Kocanaogullari, Murat Akçakaya, Deniz ErdogmusarXiv 2020 · arXiv

Introduction

Active Recursive Bayesian Inference (RBI) is a fundamental process for updating beliefs about hidden states through a sequence of informative queries. While standard methods like Maximum Mutual Information (MMI) are effective in many scenarios, they often struggle when faced with "adversarial priors"—initial beliefs that strongly favor incorrect states. This paper proposes a unified framework based on Rényi information measures that incorporates a novel "Momentum" term to track belief trajectories and overcome misleading initial information.

Methods: Momentum and Trajectory Analysis

The core innovation of this work is the integration of posterior trajectory analysis into the query selection process. By utilizing Rényi entropy and $\alpha$-divergence, the authors create a mathematically consistent framework where inference and query selection are unified.

A standout feature is the introduction of Momentum, inspired by physical dynamics. Momentum tracks the rate of change in the posterior distribution. If a state's probability is increasing rapidly, even if its absolute value remains low, Momentum encourages the system to explore it. This prevents the "zigzagging" behavior often seen in greedy methods when they encounter unexpected evidence.

Belief trajectory on a probability simplex showing the movement of posterior distribution towards a target vertex.

As shown in the figure above, the RBI process can be visualized as a particle moving across a probability simplex. The red arrows indicate the trajectory $p_0 \to p_5$, demonstrating how the belief converges toward the true target state at a vertex even when starting from a different region of the simplex.

Decision Making and Stopping Criteria

Beyond query selection, the framework addresses the decision-making process. Traditional methods often rely on a simple probability threshold to stop querying. The authors propose a "Relaxed Stopping Criterion" using I-Momentum, allowing for confident decisions when a belief is clearly "speeding" toward a specific outcome.

The geometric analysis of these decision boundaries (illustrated above) shows that entropy-based criteria create a more flexible "feasible region" (the shaded area) for making decisions compared to rigid probability thresholds ($p(a) = \tau$). This allows the system to terminate the query process more efficiently without sacrificing accuracy.

Experimental Evaluation

The framework was validated across three distinct domains:

  1. Synthetic Simulations: Monte-Carlo simulations of a target detection task demonstrated the impact of varying $\alpha$ on convergence speed.
  2. Restaurant Recommendation: Using the Entree Chicago dataset, the method showed an 8% improvement in accuracy and a 10% increase in speed over MMI in adversarial scenarios where the initial prior was incorrect.
  3. Brain-Computer Interface (BCI): In EEG-based typing tasks, the proposed method achieved a significantly higher Information Transfer Rate (ITR) and better typing accuracy than standard benchmarks.

Conclusion

By unifying query selection and decision-making through Rényi information measures and introducing trajectory-based Momentum, this framework provides a robust solution for active inference. It effectively mitigates the impact of misleading priors and offers a geometrically grounded approach to uncertainty reduction in real-world applications like recommendation systems and BCI.

Summary updated

Abstract

Recursive Bayesian inference (RBI) provides optimal Bayesian latent variable estimates in real-time settings with streaming noisy observations. Active RBI attempts to effectively select queries that lead to more informative observations to rapidly reduce uncertainty until a confident decision is made. However, typically the optimality objectives of inference and query mechanisms are not jointly selected. Furthermore, conventional active querying methods stagger due to misleading prior information. Motivated by information theoretic approaches, we propose an active RBI framework with unified inference and query selection steps through Renyi entropy and $α$-divergence. We also propose a new objective based on Renyi entropy and its changes called Momentum that encourages exploration for misleading prior cases. The proposed active RBI framework is applied to the trajectory of the posterior changes in the probability simplex that provides a coordinated active querying and decision making with specified confidence. Under certain assumptions, we analytically demonstrate that the proposed approach outperforms conventional methods such as mutual information by allowing the selections of unlikely events. We present empirical and experimental performance evaluations on two applications: restaurant recommendation and brain-computer interface (BCI) typing systems.
BibTeX
@misc{marghi2020active,
  author = {Yeganeh M. Marghi and Aziz Kocanaogullari and Murat Akçakaya and Deniz Erdogmus},
  title = {{Active Recursive Bayesian Inference with Posterior Trajectory Analysis Using α-Divergence}},
  year = {2020},
  eprint = {2004.03139},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2004.03139},
}
Figure