Embodied agents operating in complex, open-ended environments—whether navigating the digital blocks of Minecraft or manipulating physical objects with a robotic arm—face a fundamental control challenge. The real world presents a continuous, high-dimensional space of motor torques and joint angles. To make planning tractable, standard architectures split the problem hierarchically: a high-level planner (often powered by a Large Language Model) breaks a task into discrete steps, which are then mapped to a pre-defined library of low-level action primitives, such as "pick," "place," or "navigate."
As Murtaza Dalal demonstrates in Accelerating Robotic Reinforcement Learning via Parameterized Action Primitives, parameterizing these basic motions drastically simplifies exploration, shifting the agent's focus from how to move a motor to where to execute a skill. However, this abstraction introduces a critical vulnerability: the skill library bottleneck. When an agent's action space is frozen or predefined, its behavioral creativity and adaptability are strictly bounded by the foresight of its developers.
The Closed-Vocabulary Constraint
The core limitation of a fixed primitive dependency is that the agent cannot execute or synthesize any behavior missing from its executable database, regardless of its semantic intelligence. Wenlong Huang et al. identify this in Grounded Decoding: Guiding Text Generation with Grounded Models for Embodied Agents as a "closed-vocabulary constraint" that prevents robots from generating flexible, free-form plans.
This constraint creates a disconnect between perception and execution. An agent might possess the advanced visual and textual reasoning capabilities required to diagnose a situation, yet remain completely paralyzed if the necessary physical interaction was not explicitly programmed into its action heads. For example, Wenhao Chai et al. note in See and Think: Embodied Agent in Virtual Environment that if a required behavior is absent from the code library, the agent cannot interact with the environment in that specific way, despite having the perceptual data to recognize the need.
Why Composition Fails to Scale
To circumvent this rigidity, many frameworks attempt to logically or temporally compose existing primitives into more complex sequences.
- Brian Ichter (Do As I Can, Not As I Say: Grounding Language in Robotic Affordances) weights the semantic proposals of language models against a robot's physical affordances to ensure plans are realistic.
- Geraud Nangue Tasse et al. (Skill Machines: Temporal Logic Skill Composition in Reinforcement Learning) use linear temporal logic to chain base skills together zero-shot.
While these scheduling and composition mechanisms optimize how skills are ordered, they cannot synthesize fundamentally new physical interactions. If a task demands an unmodeled fine-motor adjustment—such as twisting a jar lid or stirring a liquid—the underlying composition algebra breaks down. As Kevin Lin et al. observe in Text2Motion: From Natural Language Instructions to Feasible Plans, if a complex task requires a novel manipulation primitive not included in the library, the system lacks the ability to synthesize the required motion on the fly.
This structural dependency ultimately limits long-horizon robustness. Zaijing Li et al. argue in Optimus-2: Multimodal Minecraft Agent with Goal-Observation-Action Conditioned Policy that relying on inherited action architectures or frozen action heads restricts an agent's fine-motor skills and behavioral creativity. Furthermore, traditional simulator development typically follows a bottom-up approach where researchers pre-define a narrow set of actions before attempting tasks. As Jiefu Ou et al. demonstrate in WorldAPIs: The World Is Worth How Many APIs? A Thought Experiment, this methodology forces agents to operate within narrow, often arbitrary constraints, creating a massive capability gap between hand-crafted action spaces and the open-ended complexity of the real world.
Shifting to Open-Vocabulary Action Spaces
To break free from the constraints of fixed primitive dependencies, the research direction is shifting toward systems that can dynamically expand, learn, or bypass discrete skill libraries entirely.
1. Recursive and Interactive Skill Acquisition
Instead of treating the skill library as a static database, agents can learn new primitives interactively through human collaboration or recursive definition. Matthias Scheutz et al. (Recursive Spoken Instruction-Based One-Shot Object and Action Learning) present an architecture capable of defining entirely new, complex action sequences by building recursively on top of its existing base primitives from a single spoken instruction.
Expanding this to continuous, real-time scenarios, Jennifer Grannen et al. introduce Vocal Sandbox: Continual Learning and Adaptation for Situated Human-Robot Collaboration. By moving away from static libraries, their framework allows humans and robots to co-adapt during long-horizon tasks, learning low-level physical skills on the fly through speech and gestures to prevent complete system failure when encountering unmodeled interactions.
2. Continuous Spatial Representations
Another powerful alternative is to bypass text-based or discrete skill labels entirely, mapping high-level reasoning directly to continuous spatial coordinates.
[High-Level LLM Planner]
│
▼ (Generates code describing spatial constraints)
[Continuous 3D Value Maps]
│
▼ (Low-level controller tracks optimal gradients)
[Continuous Trajectory Execution] (No discrete skill labels used)
As Wenlong Huang et al. demonstrate in VoxPoser: Composable 3D Value Maps for Robotic Manipulation with Language Models, language models can be used to synthesize continuous 3D value maps directly from observation spaces. This allows the robot to generate novel trajectories for arbitrary manipulation tasks without ever needing to reference a pre-defined primitive library.
3. End-to-End Policy Mapping
Ultimately, overcoming the skill library bottleneck requires moving toward architectures that can dynamically bridge the gap between abstract reasoning and raw physical execution. As Bin Hu 0001 et al. suggest in Enabling Intelligent Interactions between an Agent and an LLM: A Reinforcement Learning Approach, future autonomous systems must transition toward end-to-end learning of the mapping between high-level plans and raw physical actions, dissolving the rigid boundaries of discrete skill sets in favor of fluid, task-specific motor control.