Abstract
For robots to effectively integrate into human-centric environments, they must exhibit expressive behaviors that communicate intent and adhere to social norms. This paper introduces GenEM (Generative Expressive Motion), a framework that leverages the social reasoning and code-generation capabilities of Large Language Models (LLMs) to create nuanced robot behaviors from natural language instructions. By decomposing the generation process into a modular pipeline—reasoning about social norms, mapping them to specific robot hardware, and synthesizing executable code—GenEM produces adaptable motions that rival those designed by professional animators.
Introduction
Traditional approaches to robot behavior design often fall into two categories: rule-based systems, which are brittle and difficult to scale to new social contexts, and data-driven methods, which require massive, specialized datasets for every gesture. GenEM addresses these limitations by tapping into the vast "social knowledge" embedded in LLMs.
The GenEM Pipeline
The core of GenEM is a modular LLM-based pipeline that avoids the lack of interpretability found in end-to-end "black box" generation. The process follows four distinct stages:
- Expressive Instruction Following: Given a scenario, the LLM uses Chain-of-Thought (CoT) reasoning to determine the appropriate human social response (e.g., identifying that eye contact and nodding are polite when someone walks by).
- Human-to-Robot Mapping: The system translates human actions into the specific capabilities of the robot, mapping abstract concepts like a "nod" to specific hardware APIs.
- Code Generation: The LLM synthesizes Python code that calls the robot’s skill library, allowing for composable behaviors built from simpler primitives.
- Propagating Human Feedback: The system supports iterative refinement, where natural language corrections from a human (e.g., "make the nod more subtle") are used by the LLM to update the generated code.
Evaluation and Key Results
GenEM was evaluated using a benchmark of 10 behaviors ranging from simple nods to complex multi-stage interactions. The system was tested across different embodiments, including a mobile manipulator and a quadruped robot, demonstrating its embodiment-agnostic nature.
Key findings include:
- Social Reasoning as a Prior: LLMs effectively act as a repository of human social norms, reducing the need for hard-coded social logic.
- Superiority of Modularity: Breaking the task into Reasoning -> Mapping -> Coding significantly improved success rates compared to end-to-end prompting. For instance, the modular version consistently implemented necessary distance checks for social behaviors that the end-to-end version ignored.
- Human-Level Performance: In user studies, behaviors generated by GenEM with minimal human feedback were rated as highly competent, sometimes outperforming hand-crafted animations created by professional character animators.
Discussion and Future Work
While GenEM demonstrates the potential for LLMs to democratize high-quality robot animation, several areas for future research remain. The current study focused on "one-off" expressive acts; extending this to long-term, multi-turn interactions is a logical next step. Additionally, while the system handles head and base movements well, expanding the action space to include complex arm and gripper manipulations for expression would further enhance robot social capabilities. Future work will also explore the impact of real-world physical proximity on human perception of these generated behaviors.
Summary updated