Active Inference for Integrated State-Estimation, Control, and Learning

Mohamed Baioumy, Paul Duckworth, Bruno Lacerda, Nick Hawes

Abstract

This paper introduces a unified framework for robotic state estimation, control, and online learning grounded in the neuroscience-inspired theory of Active Inference. By minimizing Variational Free Energy (VFE), the proposed Active Inference Controller (AIC) achieves simultaneous estimation and control. The core contributions include the introduction of a tunable temporal parameter $\tau$ and an online hyperparameter learning scheme. Experimental results on a Franka Emika Panda manipulator demonstrate that this approach is more robust to initial parameter settings and more adaptive to varying payloads than traditional Active Inference implementations, bridging the gap between theoretical neuroscience and practical engineering.

Introduction

In classical robotics, state estimation and control are typically treated as decoupled processes. While effective in structured environments, this separation often fails when robots encounter "unmodeled dynamics"—unexpected environmental changes like varying surface friction or unknown object weights. Active Inference offers a compelling alternative by treating both perception and action as a single optimization problem: the minimization of Variational Free Energy (VFE), a measure of the discrepancy between internal models and sensory reality.

Despite its theoretical elegance, applying Active Inference to robotics has been challenging. Existing controllers are often difficult to tune, sensitive to initial conditions, and prone to oscillations. This research addresses these hurdles by formalizing the relationship between Active Inference and classical control and introducing mechanisms for real-time adaptation.

Methods

The researchers propose an Active Inference Controller (AIC) that performs state estimation, control, and learning simultaneously by minimizing VFE.

The Temporal Parameter $\tau$

A central innovation is the introduction of a temporal parameter, $\tau$, into the robot's internal generative model. This parameter dictates the time scale of the robot's beliefs. The authors provide a theoretical mapping showing that AIC is a generalization of classical methods:

  • As $\tau \to 0$, the system behaves like a standard PID controller.
  • As $\tau \to \infty$, it functions as a pure state estimator or filter.
Control behavior for different values of $\tau^{-1}$ showing how the parameter influences stability and convergence speed.

Online Hyperparameter Learning

To overcome the sensitivity of previous Active Inference models, the framework utilizes gradient descent to update "precision matrices" (analogous to controller gains) and the $\tau$ parameter online. This allows the robot to "self-tune" its dynamics in response to the task at hand, effectively damping oscillations as it approaches a target.

Experimental Setup

The framework was validated using two primary setups:

  1. Mass-Spring-Damper Simulation: Used to demonstrate the fundamental physics of the $\tau$ parameter.
  2. Franka Emika Panda Manipulator: A 7-Degree-of-Freedom (DoF) robot arm tested in both the Gazebo simulator and on real hardware.
The Franka Emika Panda robot arm used for validation in both real-world and simulated environments.

The robot performed pick-and-place operations with varying payloads (up to 3kg) to evaluate its ability to adapt to unmodeled mass changes.

Results and Discussion

The proposed AIC demonstrated significant improvements in robustness and performance:

  • Adaptive Performance: When initialized with sub-optimal parameters, the online update mechanism reduced the Mean Absolute Error (MAE) from 0.088 to 0.025—a 70% improvement.
  • Payload Handling: The system successfully managed a 3kg payload (the arm's maximum capacity) without manual retuning, maintaining smooth trajectories where static controllers failed.
  • Control via Biased Beliefs: A key insight is that control is achieved by intentionally biasing the robot's belief of its current state toward the goal. The resulting "error" drives the motor torques.

Limitations

A notable trade-off is the loss of physical meaning in the state estimate; because the belief is intentionally biased for control, it no longer accurately reflects the robot's true physical position. Furthermore, learned precision matrices lose their interpretation as sensor noise levels as they absorb control-related errors.

Conclusion

By integrating a tunable temporal scale and online learning, this work transforms Active Inference from a theoretical curiosity into a robust tool for robotic control. Future research will explore the integration of Factor Graphs to enable long-term planning within this unified framework.

Summary updated

Abstract

This work presents an approach for control, state-estimation and learning model (hyper)parameters for robotic manipulators. It is based on the active inference framework, prominent in computational neuroscience as a theory of the brain, where behaviour arises from minimizing variational free-energy. The robotic manipulator shows adaptive and robust behaviour compared to state-of-the-art methods. Additionally, we show the exact relationship to classic methods such as PID control. Finally, we show that by learning a temporal parameter and model variances, our approach can deal with unmodelled dynamics, damps oscillations, and is robust against disturbances and poor initial parameters. The approach is validated on the `Franka Emika Panda' 7 DoF manipulator.
BibTeX
@inproceedings{baioumy2020active,
  author = {Mohamed Baioumy and Paul Duckworth and Bruno Lacerda and Nick Hawes},
  title = {{Active Inference for Integrated State-Estimation, Control, and Learning}},
  booktitle = {IEEE International Conference on Robotics and Automation},
  year = {2020},
  doi = {10.1109/icra48506.2021.9562009},
}
Figure