Kinematically Grounded Planning for Embodied Agents

29 concepts25 papers1 proposal

A robot tasked with "picking up a cup" faces a two-fold challenge. First, it must logically understand the task—identifying the cup and the sequence of actions required. Second, it must physically execute those actions within the constraints of its own body. While modern Large Language Models (LLMs) and Vision-Language Models (VLMs) excel at the former, they often fail at the latter, proposing plans that are logically sound but physically impossible. Recent research focuses on bridging this gap by integrating kinematic reachability and physical constraints directly into robotic reasoning and generative models.

The Semantic-Physical Disconnect

High-level planners often suffer from a lack of "embodiment awareness." As Werner et al. and Devarakonda et al. observe, models like GPT-4 can generate sophisticated task sequences that fail because they ignore robot-specific geometry, such as joint limits, kinematic singularities, or workspace boundaries. For instance, a VLM might identify a valid semantic region for placing an object, but as Kembhavi et al. point out, that region may be entirely unreachable for the specific robotic hardware in use.

This disconnect is not limited to text-based reasoning. In grasp synthesis, Mousavian et al. note that generating a high-quality 6-DOF grasp pose does not guarantee the manipulator can physically reach those coordinates. Similarly, Kai Mo demonstrates that in complex tasks like cloth folding, predicting spatial coordinates without considering the robot's reach can result in targets that lie outside the accessible workspace.

Mapping the Reachable Workspace

To solve this, researchers are developing methods to give models an inherent sense of a robot's physical limits. Zhou et al. introduced the Space-Physical Reachability Map (S-P Map), a robot-agnostic representation that allows VLMs to distinguish between objects that are visually present and those that are kinematically reachable. By training on these maps, models can filter out impossible actions before they are attempted.

Another approach involves integrating hardware descriptions directly into the learning process. Shao et al. developed UniGrasp, which uses a robot’s Unified Robot Description Format (URDF) as an input. This allows the model to predict contact points that are inherently reachable and collision-free for any multi-fingered hand, rather than relying on hand-specific training.

Introspection and Constraint Integration

Rather than treating physical constraints as a post-hoc filter, newer frameworks embed them into the decision-making loop. Devarakonda et al. propose an "introspective" dialogue where an analyzer agent uses a physics simulator to catch kinematic errors in LLM-generated plans. Similarly, Yuecheng Liu et al. utilize reinforcement learning to penalize physically impossible plans, forcing the model to respect joint torque limits and degrees of freedom during the reasoning process.

This grounding is essential for safety and efficiency. Joshi et al. argue that without mechanisms to filter commands based on feasibility, autonomous systems like drones may attempt maneuvers that exceed their mechanical limits. Soojin Jeong further emphasizes that task planners must integrate hardware-specific affordances—such as a mobile base's movement range—to ensure that subgoals are actually executable.

Beyond Geometry: Dynamics and Stability

Physical grounding extends beyond simple reachability to include the laws of motion and structural integrity. Yong-jin Liu et al. highlight that neglecting gravity and energy costs in modular robot reconfiguration leads to paths that are physically unfeasible. In architectural construction, Chalvatzaki et al. show that plans must account for both robot kinematics and the structural stability of the assembly to prevent collapse.

By moving from purely statistical or semantic models to "graybox" approaches—which Rath et al. describe as combining classical physics priors with machine learning—researchers are creating agents that understand not just what to do, but what they are physically capable of doing. As Fu et al. demonstrate with hexapod robots, integrating transition feasibility models ensures that complex movements in simulation translate successfully to the real world.

Key Foundational Work

Learning Manipulation Skills Via Hierarchical Spatial Attention (2019) — contributed 1 concept elements to this direction.
6-DOF GraspNet: Variational Grasp Generation for Object Manipulation. (2019) — contributed 1 concept elements to this direction.
Path planning for self-reconfigurable modular robots: a survey (2017) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_physical_constraint_violation_kinematic__4e3f2fc1,
  title = {physical constraint violation (kinematic reachability constraints)},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/physical-constraint-violation-kinematic-reachability-constraints/rdr_4e3f2fc14ef94b5fb4fc6fdcfd3a6782},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-21},
}

This exact version (version 1, frozen text):

@misc{lacuna_physical_constraint_violation_kinematic__4e3f2fc1_v1,
  title = {physical constraint violation (kinematic reachability constraints)},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/physical-constraint-violation-kinematic-reachability-constraints/rdr_4e3f2fc14ef94b5fb4fc6fdcfd3a6782/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-21},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake