Reactive Temporal Logic-based Planning and Control for Interactive Robotic Tasks

Farhad Nawaz, Shaoting Peng, Lars Lindemann, Nadia Figueroa, Nikolai Matni

Abstract

Robots interacting with humans must be safe, reactive and adapt online to unforeseen environmental and task changes. Achieving these requirements concurrently is a challenge as interactive planners lack formal safety guarantees, while safe motion planners lack flexibility to adapt. To tackle this, we propose a modular control architecture that generates both safe and reactive motion plans for human-robot interaction by integrating temporal logic-based discrete task level plans with continuous Dynamical System (DS)-based motion plans. We formulate a reactive temporal logic formula that enables users to define task specifications through structured language, and propose a planning algorithm at the task level that generates a sequence of desired robot behaviors while being adaptive to environmental changes. At the motion level, we incorporate control Lyapunov functions and control barrier functions to compute stable and safe continuous motion plans for two types of robot behaviors: (i) complex, possibly periodic motions given by autonomous DS and (ii) time-critical tasks specified by Signal Temporal Logic (STL). Our methodology is demonstrated on the Franka robot arm performing wiping tasks on a whiteboard and a mannequin that is compliant to human interactions and adaptive to environmental changes.

Introduction

Robots operating in human-centric environments face a "dual-speed" challenge. They require high-level logic to follow complex instructions—such as prioritizing a specific task when a sensor triggers—while simultaneously needing low-level reactivity to maintain safety during physical human-robot interaction. Existing solutions often struggle to balance these needs: formal methods provide safety but are computationally intensive and rigid, while end-to-end learning offers flexibility but lacks formal guarantees.

This paper addresses this gap by introducing a framework that is computationally efficient enough for real-time use (1 kHz) while maintaining formal guarantees that the robot will satisfy its logic-based task specifications.

Methodology

The proposed modular architecture is split into two primary layers to handle the discrete and continuous aspects of robotic control.

Discrete Task Planner

Operating at 200 Hz, this layer utilizes Reactive Temporal Logic (RTL). It distinguishes between controllable propositions (robot actions) and uncontrollable propositions (environmental events). The planner constructs a Büchi automaton to represent task logic and employs a breadth-first search to determine the optimal path for task completion based on real-time sensor data.

Continuous Motion Planner

Operating at 1 kHz, this layer translates high-level behaviors into motor commands using several key components:

  • Dynamical Systems (DS): Defines the shape of motions, such as periodic wiping patterns.
  • Control Lyapunov Functions (CLFs): Ensures path stability.
  • Control Barrier Functions (CBFs): Guarantees safety and satisfies time-critical Signal Temporal Logic (STL) constraints.
The proposed modular control architecture integrates high-level task planning with low-level motion control.

The system is formulated as a Quadratic Program (QP), allowing for extremely fast solving and high-frequency updates. A "mixing strategy" is used to smoothly blend velocity commands when switching between tasks, preventing jerky movements.

Experimental Results

The methodology was validated using a Franka Emika Panda robot arm. Nominal motions were learned from human demonstrations using Neural Ordinary Differential Equations (Neural ODEs).

Whiteboard Scenario

The robot successfully navigated a whiteboard wiping task, switching between left and right wiping patterns. It demonstrated reactivity by prioritizing a blue stain within a 5-second deadline and responding to a "dropped eraser" event by stopping to retrieve the tool.

Mannequin Scenario

In a more complex environment, the robot performed wiping tasks on a mannequin. It demonstrated sophisticated logic by only wiping the mannequin's back upon a human gesture and periodically interrupting its task to "dip the towel in water" every 30 seconds to maintain moisture.

The mannequin scenario demonstrates the robot's ability to handle complex interactive logic and environmental changes.

Conclusion and Future Work

The framework demonstrates high-speed reactivity and formal safety. By separating controllable and uncontrollable events, the system can instantly re-route through its task automaton to recover from environmental perturbations. Furthermore, the use of CLFs and CBFs ensures the robot remains compliant and safe during human contact.

Future work will focus on handling potential deadlocks (e.g., unreachable objects) and extending the planning framework to more complex mathematical spaces like $SO(3)$ and $SE(3)$ for advanced tool manipulation. There is also potential to integrate Large Language Models (LLMs) to translate natural language commands into the RTL specifications used by this architecture.

Summary updated

Abstract

Robots interacting with humans must be safe, reactive and adapt online to unforeseen environmental and task changes. Achieving these requirements concurrently is a challenge as interactive planners lack formal safety guarantees, while safe motion planners lack flexibility to adapt. To tackle this, we propose a modular control architecture that generates both safe and reactive motion plans for human-robot interaction by integrating temporal logic-based discrete task level plans with continuous Dynamical System (DS)-based motion plans. We formulate a reactive temporal logic formula that enables users to define task specifications through structured language, and propose a planning algorithm at the task level that generates a sequence of desired robot behaviors while being adaptive to environmental changes. At the motion level, we incorporate control Lyapunov functions and control barrier functions to compute stable and safe continuous motion plans for two types of robot behaviors: (i) complex, possibly periodic motions given by autonomous DS and (ii) time-critical tasks specified by Signal Temporal Logic~(STL). Our methodology is demonstrated on the Franka robot arm performing wiping tasks on a whiteboard and a mannequin that is compliant to human interactions and adaptive to environmental changes.
BibTeX
@inproceedings{nawaz2024reactive,
  author = {Farhad Nawaz and Shaoting Peng and Lars Lindemann and Nadia Figueroa and Nikolai Matni},
  title = {{Reactive Temporal Logic-based Planning and Control for Interactive Robotic Tasks}},
  booktitle = {IEEE/RSJ International Conference on Intelligent Robots and Systems},
  year = {2024},
  doi = {10.1109/iros58592.2024.10802335},
}
Figure