ReAcTree: Hierarchical LLM Agent Trees with Control Flow for Long-Horizon Task Planning

Jae-Woo Choi, Hyungmin Kim, Hyobin Ong, Minsu Jang, Dohyung Kim, Jaehong Kim, Youngwoo Yoon

Introduction

Large Language Models (LLMs) have demonstrated remarkable reasoning capabilities, yet they often struggle as embodied agents in complex, long-horizon scenarios. Traditional frameworks, such as ReAct, typically rely on a "monolithic trajectory"—a single, linear history of observations and actions. As tasks grow in complexity (e.g., multi-room object manipulation), this linear context becomes a bottleneck, leading to hallucinations, context overflow, and a lack of structured error recovery.

To address these challenges, we present ReAcTree, a hierarchical planning framework that dynamically constructs a tree of subgoals. By decomposing complex tasks into manageable nodes and employing structured control flows, ReAcTree enables agents to maintain focus and recover from environmental failures more effectively.

ReAcTree dynamically builds a tree of subgoals to handle complex tasks, utilizing episodic and working memory to guide agent nodes.

The ReAcTree Framework

The core of ReAcTree is the dynamic expansion of a task tree, consisting of two primary types of nodes:

  1. Agent Nodes: Each node is an LLM-powered agent responsible for a specific subgoal. If a task is deemed too complex, the agent can "Expand" the tree, proposing new sub-subgoals. This semantic isolation prevents "reasoning rot" by keeping the context for each decision focused and clean.
  2. Control Flow Nodes: Inspired by robotics Behavior Trees, these nodes coordinate the execution of subgoals:
  3. Sequence ($→$): Executes subgoals in a strict order.
  4. Fallback (?): Provides a mechanism for backtracking; if one subgoal fails, the agent attempts the next alternative.
  5. Parallel ($⇒$): Allows for the simultaneous execution of independent tasks.
The ReAcTree architecture separates reasoning and acting within Agent Nodes and uses Control Flow Nodes to manage task execution logic.

The system is further supported by a dual-memory architecture. Episodic Memory stores successful subgoal trajectories for retrieval-augmented generation, while Working Memory acts as a shared blackboard for recording environmental state (e.g., object locations), reducing redundant exploration.

Evaluation and Key Insights

ReAcTree was evaluated on two major embodied AI benchmarks: WAH-NL (VirtualHome) for long-horizon household tasks and ALFRED (AI2THOR) for single-room interactions. Both environments were set to be partially observable to test the agents' ability to search and reason under uncertainty.

Several key insights emerged from the study:

  • Structure Over Scale: Smaller models (e.g., 7B parameters) using the ReAcTree framework can outperform much larger monolithic models (70B+). This suggests that architectural structure is often more critical than raw parameter count for complex planning.
  • Robustness via Fallback: The "Fallback" logic proved essential in partially observable environments, allowing agents to systematically search different locations without getting trapped in logical loops.
  • Efficiency: By bounding the context of each node to its specific subgoal, ReAcTree maintains predictable token usage and avoids the "lost in the middle" phenomenon common in long-context prompts.

Results

The framework demonstrated substantial performance improvements, particularly on the challenging WAH-NL dataset. Using Qwen 2.5 72B, ReAcTree achieved a 61% Goal Success Rate (GSR), nearly doubling the 31% success rate of a standard ReAct agent equipped with working memory. Furthermore, the Subgoal Success Rate (SSR) improved from 54% to nearly 80%. These gains were consistent across seven different LLMs, including LLaMA 3.1, Mistral, and Gemma 2, highlighting the model-agnostic nature of the framework.

Conclusion and Future Work

ReAcTree represents a significant step toward more reliable embodied AI by introducing hierarchical structure and formal control flows to LLM planning. While the framework significantly improves success rates, challenges remain regarding initial task decomposition errors and handling vague user instructions. Future research will explore "closed-loop" expansion, allowing the tree to dynamically prune or re-grow branches based on real-time environmental feedback and self-correction mechanisms.

Summary updated

Abstract

Recent advancements in large language models (LLMs) have enabled significant progress in decision-making and task planning for embodied autonomous agents. However, most existing methods still struggle with complex, long-horizon tasks because they rely on a monolithic trajectory that entangles all past decisions and observations, attempting to solve the entire task in a single unified process. To address this limitation, we propose ReAcTree, a hierarchical task-planning method that decomposes a complex goal into more manageable subgoals within a dynamically constructed agent tree. Each subgoal is handled by an LLM agent node capable of reasoning, acting, and further expanding the tree, while control flow nodes coordinate the execution strategies of agent nodes. In addition, we integrate two complementary memory systems: each agent node retrieves goal-specific, subgoal-level examples from episodic memory and shares environment-specific observations through working memory. Experiments on the WAH-NL and ALFRED datasets demonstrate that ReAcTree consistently outperforms strong task-planning baselines such as ReAct across diverse LLMs. Notably, on WAH-NL, ReAcTree achieves a 61% goal success rate with Qwen 2.5 72B, nearly doubling ReAct's 31%.
BibTeX
@misc{choi2025reactree,
  author = {Jae-Woo Choi and Hyungmin Kim and Hyobin Ong and Minsu Jang and Dohyung Kim and Jaehong Kim and Youngwoo Yoon},
  title = {{ReAcTree: Hierarchical LLM Agent Trees with Control Flow for Long-Horizon Task Planning}},
  year = {2025},
  eprint = {2511.02424},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2511.02424},
}
Figure