When a human prepares a meal or cleans a room, they operate across an extended temporal horizon. They seamlessly chain together dozens of individual actions—opening a cabinet, retrieving a bowl, scooping ingredients, and washing a utensil—while constantly adapting to small changes in their environment.
For modern robotic systems, however, this remains a profound challenge. While deep reinforcement learning and imitation learning have mastered short-horizon, atomic primitives (such as picking up a single block or grasping a door handle), scaling these capabilities to long-horizon, multi-stage tasks introduces a fundamental bottleneck: compounding errors and the combinatorial explosion of action spaces.
If a robot has a 95% success rate on a single atomic action, its probability of successfully executing a ten-step sequence drops to less than 60%. To build autonomous agents capable of operating in the real world, researchers are shifting away from monolithic end-to-end controllers toward architectures that can reason, plan, and maintain state over extended horizons.
The Horizon Bottleneck
Why do standard robotic policies fail when a task takes too long? The core issue lies in the transition between sub-goals and the maintenance of internal state.
Standard benchmarks like RLBench, developed by Stephen James et al., and ALFRED, introduced by Mohit Shridhar, explicitly expose this limitation. In these environments, an agent might successfully navigate to a refrigerator or grasp a cup, but it fails to complete the entire sequence because it cannot robustly transition from navigation to manipulation, or it loses track of the overarching objective.
As Sanjay Haresh et al. demonstrate in ClevrSkills, even when robotic policies are fine-tuned on specific multi-step sequences, their success rates plummet as the task complexity and horizon length increase. The model's attention and memory degrade over time, leading to "myopic" behavior where the robot repeats actions or fails to adapt when an intermediate step deviates from the expected outcome.
Hierarchical Decomposition
To bypass the compounding errors of end-to-end control, the prevailing strategy is hierarchical decomposition. Instead of forcing a single neural network to map raw pixels directly to low-level joint velocities over a 10-minute task, the problem is split into two distinct layers:
- A High-Level Planner: Responsible for semantic, combinatorial reasoning (e.g., deciding what sub-task to execute next).
- A Low-Level Controller: Responsible for short-duration, reactive physical execution (e.g., deciding how to move the joints to execute that specific sub-task).
This separation of concerns drastically reduces the search space. The low-level controller only needs to remain stable for a few seconds at a time, while the high-level planner operates in an abstract space where continuous physics are stripped away.
For example, in the context of complex workspace management, Amisha Bhaskar et al. developed LAVA, a hierarchical framework designed for the long-horizon task of clearing food containers. By combining high-level strategic selection with mid-level visual refinement, the system breaks down a continuous, highly unpredictable task into discrete, manageable phases. The low-level imitation learning policies are only invoked once the workspace has been strategically organized, ensuring they operate within their localized zones of high confidence.
Grounding Plans in Physical Affordances
While hierarchical decomposition solves the planning horizon problem, it introduces a new vulnerability: the gap between semantic intent and physical reality. If a high-level planner (such as a Large Language Model) dictates a sequence of steps based purely on textual logic, it may propose actions that are physically impossible given the robot's current state or surroundings—such as "pour the milk" before the milk carton has been opened.
To bridge this gap, high-level reasoning must be actively constrained by low-level physical capabilities. This is the core mechanism behind the SayCan framework (Do As I Can, Not As I Say: Grounding Language in Robotic Affordances), authored by Brian Ichter et al.
High-Level LLM ("Say") Low-Level Affordance ("Can")
[Probability of task progress] x [Probability of physical success]
\ /
\ /
v v
Combined Probability Score for Each Primitive
|
v
Selection of Next Action Step
Instead of allowing the language model to generate an entire plan in a vacuum, the system evaluates two distinct probabilities for every possible action primitive at each step of the execution loop:
- Semantic Relevance ($P_{say}$): The probability that a given action will move the robot closer to fulfilling the user's natural language instruction.
- Physical Affordance ($P_{can}$): The probability that the low-level controller can successfully execute that action given the current environmental state.
By multiplying these two metrics ($P_{say} \times P_{can}$), the robot selects the action that is both useful and feasible. If the robot is instructed to bring a snack, and the snack is inside a closed drawer, the affordance for "grasp snack" will be near zero, while the affordance for "open drawer" will be high. The system automatically chains the prerequisite actions together based on real-time physical feedback, allowing it to recover gracefully if an intermediate step fails.
Scaling Temporal Memory with Sequence Models
Even with robust affordances, a long-horizon robot cannot act purely reactively; it must maintain a memory of past actions and observations to ensure global coherence.
Traditional behavior-cloning architectures treat each time step as independent, which often causes robots to get stuck in behavioral loops. To address this, researchers are leveraging sequence models capable of processing long temporal contexts. Siddhant Haldar et al. introduced BAKU, an efficient transformer architecture that uses a decoder-only trunk to model sequences of actions across multiple tasks, allowing the robot to autonomously chain sub-behaviors together over time.
However, as the execution horizon scales to hundreds or thousands of steps, the quadratic computational complexity of Transformers becomes a severe bottleneck for real-world deployment.
To achieve scalability, Bo Liu et al. proposed RoboSSM, a framework that replaces traditional Transformers with State-Space Models (SSMs) for in-context imitation learning. Because SSMs compress historical context into a fixed-size hidden state, they achieve linear complexity relative to sequence length.
This architectural shift allows robots to ingest demonstration prompts that are up to 16 times longer than those seen during training. The robot can maintain a continuous, long-term memory of its progress throughout a multi-stage task, preserving high success rates and real-time inference speeds even as the operational horizon expands.