Introduction
Large Language Models (LLMs) have demonstrated remarkable reasoning capabilities, yet they often struggle as embodied agents in complex, long-horizon scenarios. Traditional frameworks, such as ReAct, typically rely on a "monolithic trajectory"—a single, linear history of observations and actions. As tasks grow in complexity (e.g., multi-room object manipulation), this linear context becomes a bottleneck, leading to hallucinations, context overflow, and a lack of structured error recovery.
To address these challenges, we present ReAcTree, a hierarchical planning framework that dynamically constructs a tree of subgoals. By decomposing complex tasks into manageable nodes and employing structured control flows, ReAcTree enables agents to maintain focus and recover from environmental failures more effectively.
The ReAcTree Framework
The core of ReAcTree is the dynamic expansion of a task tree, consisting of two primary types of nodes:
- Agent Nodes: Each node is an LLM-powered agent responsible for a specific subgoal. If a task is deemed too complex, the agent can "Expand" the tree, proposing new sub-subgoals. This semantic isolation prevents "reasoning rot" by keeping the context for each decision focused and clean.
- Control Flow Nodes: Inspired by robotics Behavior Trees, these nodes coordinate the execution of subgoals:
- Sequence ($→$): Executes subgoals in a strict order.
- Fallback (?): Provides a mechanism for backtracking; if one subgoal fails, the agent attempts the next alternative.
- Parallel ($⇒$): Allows for the simultaneous execution of independent tasks.
The system is further supported by a dual-memory architecture. Episodic Memory stores successful subgoal trajectories for retrieval-augmented generation, while Working Memory acts as a shared blackboard for recording environmental state (e.g., object locations), reducing redundant exploration.
Evaluation and Key Insights
ReAcTree was evaluated on two major embodied AI benchmarks: WAH-NL (VirtualHome) for long-horizon household tasks and ALFRED (AI2THOR) for single-room interactions. Both environments were set to be partially observable to test the agents' ability to search and reason under uncertainty.
Several key insights emerged from the study:
- Structure Over Scale: Smaller models (e.g., 7B parameters) using the ReAcTree framework can outperform much larger monolithic models (70B+). This suggests that architectural structure is often more critical than raw parameter count for complex planning.
- Robustness via Fallback: The "Fallback" logic proved essential in partially observable environments, allowing agents to systematically search different locations without getting trapped in logical loops.
- Efficiency: By bounding the context of each node to its specific subgoal, ReAcTree maintains predictable token usage and avoids the "lost in the middle" phenomenon common in long-context prompts.
Results
The framework demonstrated substantial performance improvements, particularly on the challenging WAH-NL dataset. Using Qwen 2.5 72B, ReAcTree achieved a 61% Goal Success Rate (GSR), nearly doubling the 31% success rate of a standard ReAct agent equipped with working memory. Furthermore, the Subgoal Success Rate (SSR) improved from 54% to nearly 80%. These gains were consistent across seven different LLMs, including LLaMA 3.1, Mistral, and Gemma 2, highlighting the model-agnostic nature of the framework.
Conclusion and Future Work
ReAcTree represents a significant step toward more reliable embodied AI by introducing hierarchical structure and formal control flows to LLM planning. While the framework significantly improves success rates, challenges remain regarding initial task decomposition errors and handling vague user instructions. Future research will explore "closed-loop" expansion, allowing the tree to dynamically prune or re-grow branches based on real-time environmental feedback and self-correction mechanisms.
Summary updated