Abstract
Autonomous robots need to interact with unknown, unstructured and changing environments, constantly facing novel challenges. Therefore, continuous online adaptation for lifelong-learning and the need of sample-efficient mechanisms to adapt to changes in the environment, the constraints, the tasks, or the robot itself are crucial. In this work, we propose a novel framework for probabilistic online motion planning with online adaptation based on a bio-inspired stochastic recurrent neural network. By using learning signals which mimic the intrinsic motivation signal cognitive dissonance in addition with a mental replay strategy to intensify experiences, the stochastic recurrent network can learn from few physical interactions and adapts to novel environments in seconds. We evaluate our online planning and adaptation framework on an anthropomorphic KUKA LWR arm. The rapid online adaptation is shown by learning unknown workspace constraints sample-efficiently from few physical interactions while following given way points.
Introduction
While robots excel in structured industrial settings, the transition to unstructured, "real-world" environments remains a significant challenge. Traditional motion planning models are often static; when a robot encounters an unforeseen obstacle or experiences internal hardware changes, these models typically fail. Adapting to such changes usually requires human intervention to redefine tasks or rewards. This research addresses the need for autonomous adaptation, enabling robots to detect discrepancies between their internal models and reality and update themselves "online" during execution.
Methodology: The SRNN Framework
The proposed framework is built upon three bio-inspired pillars designed for high sample efficiency:
- Stochastic Recurrent Neural Networks (SRNN): Inspired by hippocampal "place cells," the SRNN represents the environment and potential trajectories as probability distributions. This allows the robot to sample multiple possible paths rather than relying on a single deterministic calculation.
- Cognitive Dissonance as Intrinsic Motivation: The system continuously compares its mental plan with the actual execution. When a collision or deviation occurs, the resulting discrepancy—termed "cognitive dissonance"—serves as an intrinsic learning signal. This allows the robot to identify exactly when and where its internal model is inaccurate.
- Mental Replay: To maximize learning from limited physical interactions, the robot employs a mental replay strategy. Following a single physical event, the system generates multiple neural encodings (e.g., 20 replays) to update synaptic weights. This process mimics the way biological systems consolidate memories to accelerate learning.
The researchers further distinguished between Global adaptation signals (a single error for the network) and Local signals (neuron-specific signals based on their contribution to the movement).
Experimental Setup and Results
The framework was validated using both realistic dynamic simulations and a physical KUKA LWR arm. The primary task involved navigating a 60x60 cm workspace to reach specific waypoints while avoiding previously unknown obstacles.
Key Findings:
- Sample Efficiency: The system demonstrated remarkable efficiency, adapting to novel environments in just 2.8 to 8.6 physical interactions. This is a significant improvement over traditional deep learning methods that require thousands of trials.
- Local Signal Superiority: Local adaptation signals proved more effective than global ones. Although they required slightly more interactions to trigger initially, they resulted in more natural evasive maneuvers and higher success rates in reaching targets (13.7 vs. 10.4 successful reaches within a fixed timeframe).
- Real-Time Performance: The entire learning cycle, including mental replay, was completed in approximately 43 milliseconds per segment, facilitating true online operation.
- Sim-to-Real Transfer: Models pre-trained in simulation successfully transitioned to the physical KUKA arm. Upon encountering a new obstacle (a soda can), the robot adapted its internal model within 7 interactions to successfully navigate the workspace.
Discussion and Future Work
The current framework demonstrates that combining intrinsic motivation with mental replay allows for rapid, autonomous adaptation in stochastic environments. However, the use of full population coding presents a "curse of dimensionality" as task complexity increases.
Future research will explore factorized coding to handle higher-dimensional spaces and more complex robotic configurations. Additionally, the authors aim to investigate the system's ability to recover from internal hardware failures (e.g., locked joints) and the integration of "curiosity" signals to drive proactive exploration of unknown workspace areas.
Summary updated