Traditional robotic control relies heavily on a clean, predictable world. To make a robot perform a task—whether it is a quadruped walking across a field or a robotic arm assembling a piece of machinery—engineers historically divided the problem into a sequence of discrete, hand-coded stages. For an assembly task, this might look like a state machine: Move to coordinates $\rightarrow$ Search for contact $\rightarrow$ Align parts based on a force threshold $\rightarrow$ Insert.
While this approach provides high precision under ideal conditions, it is notoriously brittle. If a sensor encounters unexpected noise, if an object slips slightly, or if the environment changes in a way the programmer did not explicitly foresee, the rigid thresholds fail to trigger, and the entire sequence collapses.
To overcome this manual control brittleness, a major research direction focuses on replacing hand-coded heuristics, rigid state transitions, and pre-defined motion primitives with adaptive control hierarchies. By decoupling high-level behavioral planning from low-level physical execution, and replacing hardcoded thresholds with continuous optimization or learned policies, these systems achieve unprecedented robustness in unpredictable, real-world environments.
The Failure Mode of Rigid Primitives
To understand why traditional systems fail, consider a robot trying to manipulate an object in a "contact-rich" scenario, such as inserting a peg into a hole or rotating an object in its hand.
In a classic framework, transitioning from "searching" for the hole to "inserting" the peg requires a hardcoded sensor threshold (e.g., if downward force exceeds $X$ Newtons, assume alignment). However, as Migimatsu et al. point out, these fixed values fail to generalize across different objects or varying physical environments. If the surface is slightly softer than expected, the force threshold might never be reached, causing the robot to press down indefinitely until it damages itself or throws an error.
Similarly, when a robot interacts dynamically with an object, it must constantly negotiate changing contact points. Traditional contact-explicit control requires pre-defining exactly when and where these contacts should occur. As Jiang et al. note, this makes the system highly sensitive to unexpected object slippage. If the object moves slightly out of place, the pre-defined contact sequence becomes invalid.
The core issue is that rigid primitives treat continuous, noisy physical interactions as a series of discrete, deterministic steps. When the physical reality deviates from the programmer's assumptions, the system has no mechanism to adapt.
Decoupling Planning from Execution via Hierarchies
The modern alternative is to structure control as a flexible hierarchy. Instead of a single program trying to manage both high-level intent and low-level motor voltages, the problem is split into two or more layers that communicate through continuous spaces rather than discrete triggers.
┌─────────────────────────────────────────────────────────┐
│ High-Level Planner (World Model / Foundation Model / RL)│
│ Outputs: Continuous goals, constraints, or target states│
└────────────────────────────┬────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────┐
│ Low-Level Controller (MPC / Reactive Tracking Policy) │
│ Outputs: High-frequency joint torques and motor actions │
└─────────────────────────────────────────────────────────┘
In this architecture, the high-level planner focuses on what to do, while the low-level controller focuses on how to physically execute it in real-time.
For example, Hansen et al. developed a hierarchical world model called Puppeteer for humanoid robots. The high-level visual planner looks at the environment (e.g., a flight of stairs) and plans a trajectory of abstract "global" movements. It does not care about individual joint torques. The low-level controller takes these abstract goals and translates them into physical motor actions. Because the layers are decoupled, the robot can navigate variations in the stairs without needing a pre-defined "stair-climbing skill primitive."
This hierarchical approach dates back to early foundational work in behavior-based robotics. Mahadevan et al. and Fuentes et al. demonstrated that breaking complex navigation down into a hierarchy of simpler, learned reactive behaviors allowed physical robots to autonomously discover robust ways to interact with their surroundings, bypassing the need to hand-code exhaustive sensor interactions.
Replacing Heuristics with Continuous Optimization
Instead of using "if-else" rules to switch between behaviors, modern frameworks use mathematical optimization and reinforcement learning (RL) to compute the best action dynamically.
Contact-Implicit Planning
Instead of pre-defining when a robot should grip, slide, or release an object, contact-implicit methods allow the system to discover these transitions automatically. Katayama et al. formulated contact-rich manipulation as a Linear Complementarity Quadratic Program (LCQP). This mathematical framework allows the robot to make real-time decisions about whether to stick, slip, or break contact based on current physics, entirely eliminating the need for heuristic switching rules. Similarly, the contact-implicit model predictive control used by Jiang et al. allows a dexterous robotic hand to recover from unexpected object slippage by recalculating contact points on the fly at 20Hz.
Proprioceptive Feedback over Explicit Estimation
In legged locomotion, traditional controllers rely on explicit contact estimators to determine if a foot has touched the ground before transitioning to the next step. On deformable or slippery surfaces like mud or snow, these estimators fail. Wellhausen et al. bypassed this brittleness by training a reinforcement learning policy that relies entirely on proprioception (internal joint and IMU data). The network learns an implicit understanding of the terrain's physics through interaction, allowing a quadruped to navigate extreme environments without any hand-engineered state machines.
Scaling to General Purpose Behaviors
By moving away from task-specific, hand-coded controllers, researchers can train single, unified models capable of executing hundreds of diverse skills.
To prevent a robot from forgetting how to perform one task when learning another, Klipfel et al. introduced an Adaptive Motion Sampling technique. This allowed a single RL policy to master over 700 diverse behaviors on a quadrupedal robot, enabling seamless transitions between vastly different motor skills. To feed these hungry learning algorithms with enough data, Fang et al. developed a framework that automatically generates a diverse curriculum of tasks, allowing robots to discover generalizable skills like pushing and inserting without manual engineering.
This scalability culminates in the concept of Behavior Foundation Models (BFMs). As surveyed by Yuan et al., BFMs leverage large-scale pre-training to learn reusable behavioral priors. Instead of writing a new controller for every new task, developers can rapidly adapt these foundation models to new scenarios.
Furthermore, these learned priors can be combined with high-level semantic reasoning tools. For instance, Strobel et al. integrated Large Language Models (LLMs) into robot swarms to handle real-time reasoning and anomaly detection, moving past the need to program expert-level rules for every anticipated swarm scenario. For fine-grained manipulation, Chen et al. utilized Vision-Language Models to extract hierarchical constraints directly from human demonstration videos, allowing robots to mimic the subtle curves and nuances of human movement without relying on rigid, pre-defined motion primitives.
Finding the Right Balance
While end-to-end learning offers maximum flexibility, completely discarding structure can make optimization impractical for high-precision tasks. The most effective implementations often combine the reliability of structured control with the adaptability of learning.
As Voigt et al. demonstrate, a multi-level structure that uses pre-defined skills to narrow the search space—paired with data-driven optimization algorithms—allows robots to solve high-precision tasks like inserting a key into a lock while maintaining the flexibility to handle environmental noise. By replacing rigid heuristic thresholds with these adaptive, optimization-driven layers, modern robotics is transitioning from fragile, deterministic execution to robust, reactive intelligence.
Key Foundational Work
Hierarchical Learning of Reactive Behaviors in an Autonomous Mobile Robot (1997) — contributed 1 concept elements to this direction.
Automatic Programming of Behavior-Based Robots Using Reinforcement Learning (1991) — contributed 1 concept elements to this direction.