Iterative Closed Loop Planning for Autonomous Agents

This direction is merged into another direction: dynamic plan adaptation. Its last version stays readable and citable here.

Version 1 of 157 papers64 conceptsfrozen 2026-10-04

In traditional robotics and AI, planning was often treated like a paper map: you plot your entire route from start to finish before you ever leave the driveway. This "open-loop" approach works perfectly in a static, predictable world. But the real world is messy—paint drips, objects get moved by humans, and software interfaces lag. When an agent follows a static plan in a dynamic environment, a single error at step two makes the remaining ninety-eight steps irrelevant.

A major shift in recent research moves away from one-shot plan generation toward Iterative Closed-Loop Planning. Instead of generating a full sequence of actions upfront, agents now treat planning as a continuous process of perception, reasoning, and adjustment.

The Fragility of Static Sequences

The fundamental problem with "open-loop" planning is that Large Language Models (LLMs) often lack a "sense" of the physical world. As noted in LASP: Surveying the State-of-the-Art in Large Language Model-Assisted AI Planning by Haoming Li, while LLMs possess vast commonsense knowledge, they struggle with physical constraints and execution errors.

If you ask a robot to "make a smiley face" using blocks, an open-loop system might generate all the coordinates at once. If the robot accidentally bumps a block out of place during the first move, the rest of the pre-calculated coordinates are now wrong. Research like AlphaBlock by Chuhao Jin addresses this by implementing CogLoop, which generates only a single sub-task at a time, re-evaluating the environment after every action to ensure the next step is grounded in current reality.

Perceive, Reason, Act, Repeat

The core mechanism of this research direction is the feedback loop. By integrating environmental signals back into the reasoning engine, the agent can "self-reflect" and correct its course.

  1. Perceive: The agent captures the current state (via vision, sensors, or API logs).
  2. Reason: The foundation model compares this state against the ultimate goal.
  3. Act: The agent executes a small, discrete step.
  4. Repeat: The agent immediately re-perceives the new state to plan the next step.

This loop is central to the SayCan framework by Brian Ichter and Fei Xia in Do As I Can, Not As I Say. By multiplying what the LLM suggests ("Say") with the probability that a skill will actually succeed in the current state ("Can"), the robot can adapt if a required object is missing or moved. Similarly, Yingdong Hu and colleagues introduced VILA in Look Before You Leap, which uses GPT-4V to re-evaluate the scene after every skill execution, allowing the robot to handle dynamic obstacles that weren't there when the task started.

Grounding Reasoning in Real-Time Feedback

The "closed-loop" isn't just about fixing errors; it’s about grounding. Grounding is the process of linking abstract linguistic concepts (like "the yellow block") to specific physical entities (pixels in a camera feed).

In Embodied Chain of Action Reasoning, Hao Huang demonstrates how humanoid robots use a "Chain-of-Action" (CoA) to bridge high-level intent with low-level constraints. By continuously updating the internal scene representation, the robot can detect if it failed to grasp an object and immediately retry or adjust its approach. This is echoed in the work of Kevin Black on SuSIE (Zero-Shot Robotic Manipulation with Pretrained Image-Editing Diffusion Models), where the system periodically "refreshes" its visual subgoals to ensure the robot's mental rehearsal stays aligned with its actual perspective.

Beyond Robots: Digital and Symbolic Agents

This iterative philosophy extends beyond physical robots into digital assistants and software agents. In Dynamic Planning for LLM-based Graphical User Interface Automation, Shaoqing Zhang introduces D-PoT, a framework for smartphone automation. Because mobile apps are unpredictable—pop-ups appear, or pages load slowly—D-PoT continuously refines its plan based on real-time screenshots.

Even in purely symbolic tasks like tool use or table reasoning, the loop is vital. Sehoon Kim and the team behind LLMCompiler (An LLM Compiler for Parallel Function Calling) use adaptive replanning cycles to evaluate the outputs of intermediate tools. If a tool returns an unexpected result, the system modifies the remaining task graph on the fly.

Why It Works: Resilience and Regret

The intuition behind why this works is best captured by Zhihan Liu in the RAFA framework (Reason for Future, Act for Now). By treating agent reasoning as a planning problem within a Markov Decision Process, RAFA provides mathematical "regret" guarantees.

The "Reason for Future" part allows the agent to look ahead, but the "Act for Now" part ensures it only commits to the immediate next step. This prevents the agent from being "hallucination-led"—where a model believes it has completed a step that actually failed—because the environment's feedback serves as an objective source of truth that overrides the model's internal assumptions. As Mingde Zhao argues in Skipper, frequent regeneration of plans ensures that even if the initial generative model produces a low-quality estimate, the agent remains robust by constantly correcting its trajectory.

Papers in this version

History & citationVersion 1 · 2026-10-04

Merged into another direction · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_iterative_closed_loop_planning_for_auton_e0ecd727,
  title = {Iterative Closed Loop Planning for Autonomous Agents},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/iterative-closed-loop-planning-for-autonomous-agents/rdr_e0ecd727c45f41c7a89d3d30e8ce7145},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-21},
}

This exact version (version 1, frozen text):

@misc{lacuna_iterative_closed_loop_planning_for_auton_e0ecd727_v1,
  title = {Iterative Closed Loop Planning for Autonomous Agents},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/iterative-closed-loop-planning-for-autonomous-agents/rdr_e0ecd727c45f41c7a89d3d30e8ce7145/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-21},
}