Hierarchical Task Decomposition for Complex Reasoning

This direction is merged into another direction: hierarchical query decomposition. Its last version stays readable and citable here.

Version 1 of 180 papers90 conceptsfrozen 2026-10-04

Large Language Models (LLMs) often suffer from a "compositional reasoning gap." While they can solve simple, atomic tasks with high accuracy, their performance collapses when those tasks are nested or chained together. As Yepeng Liu and colleagues demonstrate in ConvexBench: Can LLMs Recognize Convex Functions?, a model that perfectly understands basic mathematical operations may fail entirely when asked to evaluate a function with 100 layers of composition.

The emerging solution across research is Hierarchical Task Decomposition: a paradigm shift from linear "Chain of Thought" (CoT) processing to a recursive, tree-structured approach that partitions complex queries into manageable, independent sub-problems.

The Failure of Linear Reasoning

Standard prompting techniques like CoT encourage models to "think step-by-step." However, for highly complex tasks, this linear chain is fragile. Errors in early steps accumulate, and the "reasoning horizon"—the amount of context a model must hold in active attention—expands until the model loses focus.

Jingyuan Qi and others argue in The Art of SOCRATIC QUESTIONING: Recursive Thinking with Large Language Models that linear chains lack a mechanism for backtracking or parallel exploration. When a model encounters a multi-layered problem, such as a nested SQL query or a multi-step medical diagnosis, a single-pass approach forces it to manage the entire logic hierarchy simultaneously, leading to "stochastic parroting" rather than genuine causal reasoning.

Maintaining a Constant Reasoning Horizon

The core intuition behind hierarchical decomposition is to keep the model's cognitive load constant, regardless of the problem's total complexity. By breaking a deep problem into a sequence of shallow steps, the model only ever needs to solve "single-hop" problems.

  1. Partitioning: A "Decomposer" unit (often the LLM itself or a specialized agent) identifies the constituent parts of a query. Pruthvi Patel et al. show in Is a Question Decomposition Unit All We Need? that this restructuring can boost performance by up to 29% by leveraging the model's existing strengths on simpler components.
  2. Recursive Execution: If a sub-problem is still too complex, the system applies the decomposition recursively. This creates a "Task Forest" or "Thought Tree." In RECPARSER: A Recursive Semantic Parsing Framework for Text-to-SQL Task, Qian Liu uses this to transform nested SQL generation into a series of non-nested sub-tasks.
  3. Bottom-Up Synthesis: Once the leaf-node sub-questions are answered, the results are aggregated upward. This ensures that the final answer is grounded in verifiable intermediate facts.

Pruning and Working Memory

A significant bottleneck in long-horizon reasoning is the context window. Linear chains fill the context with "scratchpad" noise—the messy internal details of every intermediate step.

In Beyond Context Limits: Subconscious Threads for Long-Horizon Reasoning, Hongyin Luo and the MIT CSAIL team propose a "subconscious" approach: the system retains only the conclusions of completed sub-tasks and prunes their internal details. This allows models to handle thousands of reasoning steps without being overwhelmed by irrelevant historical data, effectively maintaining a focused working memory.

Domain-Specific Successes

This hierarchical shift is proving transformative across diverse fields:

  • Mathematics: Chengpeng Li 0001 introduces DotaMath, which uses "Decomposition of Thought" to allow small 7B models to outperform 70B giants by breaking problems into sub-tasks solved by a Python interpreter.
  • Medical Quality Control: The IMQC platform, described by Qi Ye, uses "least-to-most" prompting to decompose medical records into verifiable facts, achieving human-expert accuracy.
  • Scientific Analysis: Yuan Chang and colleagues developed TreeReview, which uses a "tree of questions" to probe deeply into scientific papers, reducing computational costs by 80% while increasing technical specificity.

Why It Works: The "System 2" Transition

Ultimately, hierarchical decomposition moves LLMs from "System 1" (intuitive, fast, pattern-matching) to "System 2" (deliberate, slow, logical) reasoning. As Changcheng Li notes in MTMT, task decomposition is the most critical cognitive strategy for this transition. By forcing a model to explicitly define and solve the prerequisites of a problem, we prevent it from jumping to a "hallucinated" conclusion based on surface-level similarities in the prompt.

Papers in this version

Showing 60 of 80 papers.

History & citationVersion 1 · 2026-10-04

Merged into another direction · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_hierarchical_task_decomposition_for_comp_d4c7f7c9,
  title = {Hierarchical Task Decomposition for Complex Reasoning},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/hierarchical-task-decomposition-for-complex-reasoning/rdr_d4c7f7c9f5514331b3a8bc557e386c34},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-22},
}

This exact version (version 1, frozen text):

@misc{lacuna_hierarchical_task_decomposition_for_comp_d4c7f7c9_v1,
  title = {Hierarchical Task Decomposition for Complex Reasoning},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/hierarchical-task-decomposition-for-complex-reasoning/rdr_d4c7f7c9f5514331b3a8bc557e386c34/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-22},
}