The Granularity Tradeoff in Task Decomposition

42 concepts24 papers6 proposals

When tackling complex reasoning problems, the standard recipe in machine learning is simple: divide and conquer. By breaking a massive problem down into smaller, sequential sub-tasks—as seen in Chain-of-Thought prompting or multi-agent planning—we can guide models through intricate logical pathways that they could never navigate in a single leap.

However, this strategy harbors a hidden paradox. If you decompose a problem too little, the model stumbles over a massive logical gap. But if you decompose it too much, you introduce a cascade of intermediate steps where tiny errors, hallucinations, and false positives compound exponentially.

This tension is known as the granularity-noise tradeoff. Rather than a linear relationship where finer steps always yield better results, task decomposition follows an inverted U-shaped curve. Finding the peak of this curve requires balancing the risk of logical leaps against the danger of error propagation.

Performance
    ^
    |          Optimal Granularity
    |               [Peak]
    |               /   \
    |  Logic Leaps /     \  Error Propagation &
    |  (Too Coarse)       \ Redundant Noise (Too Fine)
    +---------------------------------------->
                     Granularity (Number of Steps)

Why Over-Decomposition Fails: The Mechanics of Noise

To understand why more steps can degrade performance, we must look at how errors propagate through computational chains. When an algorithm or language model breaks a task down into highly granular sub-steps, it creates multiple points of failure.

1. Exponential Error Propagation

In a multi-step pipeline, each sub-task acts as a noisy filter. If a model has a slight tendency toward false positives or minor hallucinations, these errors accumulate at each step.

Yaliang Li et al. in On the Design and Analysis of LLM-Based Algorithms and Yanxi Chen et al. in Designing Algorithms Empowered by Language Models: An Analytical Framework, Case Studies, and Insights formalize this behavior by modeling LLM workflows as computational graphs. They demonstrate that when a model is prone to false positives, excessive decomposition causes the accumulated noise from dozens of tiny sub-tasks to completely overwhelm the final aggregation step.

Instead of simplifying the problem, over-partitioning creates an environment where small inaccuracies compound until they derail the final conclusion, a vulnerability also highlighted by Dheeru Dua et al. in Successive Prompting for Decomposing Complex Questions and Zhen Yang et al. in Triples as the Key: Structuring Makes Decomposition and Verification Easier in LLM-based TableQA.

2. Context Fragmentation

Decomposition inherently isolates information. By forcing a model to focus exclusively on an atomic sub-problem, we strip away the global context that may be necessary to resolve local ambiguities.

This phenomenon is not unique to language models; it is a fundamental property of task design. In crowdsourcing, Hao Gu et al. note in P&T-Inf: A Result Inference Method for Context-Sensitive Tasks in Crowdsourcing that while micro-tasks make work simpler for individual human workers, they remove the contextual cues required to interpret ambiguous elements correctly.

When applied to LLMs, Yebowen Hu et al. show in When Reasoning Meets Information Aggregation: A Case Study with Sports Narratives that excessive fragmentation introduces heavy instruction overhead and triggers hallucinations. The model loses the "forest" for the "trees," making it impossible to maintain coherence across long reasoning chains, a challenge echoed by Kushal Jain et al. in First-Step Advantage: Importance of Starting Right in Multi-Step Math Reasoning.

3. Global Structure Disruption

In generative and visual tasks, over-decomposition can shatter the macro-structure of the output. For example, in text-to-image synthesis, Chen-Yi Lu et al. discover in RECON: Training-Free Acceleration for Text-to-Image Synthesis with Retrieval of Concept Prompt Trajectories that breaking a prompt into too many isolated visual concepts results in a cluttered, incoherent initial state.

Similarly, in visual grounding, Joy Hsu demonstrates in What Makes a Maze Look Like a Maze? that overly complex, fine-grained conceptual schemas introduce excessive noise, overwhelming the model's ability to link symbols back to visual features.

Calibrating Granularity to Model Capacity

The optimal point on the granularity curve is not fixed—it shifts based on the inherent capacity of the model performing the task.

Performance
    ^
    |       /---\  [Smaller Models: Need fine steps]
    |      /     \
    |     /       \----\ [Larger Models: Suffer from over-decomposition]
    |    /              \
    +---------------------------------------->
                     Granularity (Number of Steps)

As Pruthvi Patel et al. observe in Is a Question Decomposition Unit All We Need?, smaller models require highly frequent, simple sub-steps because they lack the native reasoning capacity to bridge large logical gaps. Larger models, however, possess a wider reasoning horizon and can process larger, context-rich chunks directly.

Forcing a high-capacity model to use rigid, multi-level decomposition can actually degrade its performance compared to end-to-end reasoning. Yike Wu et al. demonstrate this bottleneck in MLDT: Multi-Level Decomposition for Complex Long-Horizon Robotic Task Planning with Open-Source Large Language Model, showing that structured decomposition benefits smaller models but acts as a constraint on more capable architectures.

This architectural dependency also alters how we should supervise models. Sarah Pan et al. show in Let's Reinforce Step by Step that while process-level rewards (reinforcing every intermediate step) significantly boost performance on simple math problems, they hinder models on complex problems. Fine-grained step rewards introduce unnecessary constraints and noise when a problem allows for multiple valid, holistic solution paths.

Managing the Tradeoff

Because the optimal level of decomposition depends heavily on the task's difficulty and the model's intelligence, static workflows often fail. In automated planning, Chih-Wei Hsu et al. noted in Constraint Partitioning for Solving Planning Problems with Trajectory Constraints and Goal Preferences that while large partitions are computationally intractable, excessively fine-grained partitions generate a massive volume of global conflicts that are difficult to resolve.

To navigate this, modern reasoning frameworks are moving toward dynamic and controllable decomposition:

Ultimately, task decomposition is not a silver bullet that scales infinitely with finer steps. It is a delicate optimization problem where the goal is to decompose just enough to make a problem solvable, but no further.

Key Foundational Work

Constraint Partitioning for Solving Planning Problems with Trajectory Constraints and Goal Preferences (2007) — contributed 1 concept elements to this direction.
Hybrid Qualitative Simulation of Military Operations (2011) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_decomposition_granularity_tradeoff_0f4c9c3d,
  title = {decomposition granularity tradeoff},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/decomposition-granularity-tradeoff/rdr_0f4c9c3de6bf408d8f8198c6d0823ab7},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_decomposition_granularity_tradeoff_0f4c9c3d_v1,
  title = {decomposition granularity tradeoff},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/decomposition-granularity-tradeoff/rdr_0f4c9c3de6bf408d8f8198c6d0823ab7/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake