LLM⨉MapReduce-V2: Entropy-Driven Convolutional Test-Time Scaling for Generating Long-Form Articles from Extremely Long Resources

Haoyu Wang, Yujia Fu, Zhu Zhang, Shuo Wang, Zirui Ren, Xiaorong Wang, Zhili Li, Chaoqun He, Bo An, Zhiyuan Liu, Maosong SunarXiv 2025 · OpenReview

Abstract

Long-form generation is crucial for a wide range of practical applications, typically categorized into short-to-long and long-to-long generation. While short-to-long generations have received considerable attention, generating long texts from extremely long resources remains relatively underexplored. The primary challenge in long-to-long generation lies in effectively integrating and analyzing relevant information from extensive inputs, which remains difficult for current large language models (LLMs). In this paper, we propose LLM$ imes$MapReduce-V2, a novel test-time scaling strategy designed to enhance the ability of LLMs to process extremely long inputs. Drawing inspiration from convolutional neural networks, which iteratively integrate local features into higher-level global representations, LLM$ imes$MapReduce-V2 utilizes stacked convolutional scaling layers to progressively expand the understanding of input materials. Both quantitative and qualitative experimental results demonstrate that our approach substantially enhances the ability of LLMs to process long inputs and generate coherent, informative long-form articles, outperforming several representative baselines.

Introduction: Beyond Extractive Retrieval

In the era of long-context Large Language Models (LLMs), the challenge of "long-to-long" generation—such as creating a 10,000-word academic survey from thousands of pages of research—remains a significant hurdle. Current approaches often rely on Retrieval-Augmented Generation (RAG), which primarily functions as an extractive process. RAG identifies specific snippets that match a query but frequently overlooks nuanced connections and broader context hidden in the "non-matching" portions of the text.

Comparison between traditional extractive methods and the proposed integrative approach.

When faced with extremely long inputs, LLMs often suffer from the "lost in the middle" phenomenon or fail to synthesize a cohesive narrative. LLM×MapReduce-V2 addresses these bottlenecks by shifting from an extractive paradigm to an integrative one, treating massive document collections as a hierarchical structure to be synthesized rather than a database to be queried.

The LLM×MapReduce-V2 Framework

The framework introduces a "test-time scaling" strategy inspired by Convolutional Neural Networks (CNNs). Just as a CNN aggregates local pixels into textures and then into objects, LLM×MapReduce-V2 aggregates local information into higher-level global representations through three main stages:

1. Initialization

The process begins by creating a "Survey Tree" (a skeleton or outline) and "Digests"—compressed summaries of input papers tailored specifically to that skeleton. This ensures that the initial data reduction is guided by the desired final structure.

2. Entropy-Driven Convolutional Scaling

This stage is the core innovation. The framework uses stacked convolutional layers to iteratively integrate local feedback into global representations.

  • Randomized Convolutional Layers: Instead of attempting to process all data simultaneously, the system samples sets of feedback items.
  • Entropy-Driven Sampling: To optimize quality, the system uses an "LLM-as-judge" to estimate the information entropy (informativeness) of different skeleton versions. It prioritizes paths that maximize this entropy, ensuring the final representation is as informative as possible.
The LLM×MapReduce-V2 architecture, illustrating the convolutional scaling layers and entropy-driven sampling process.

3. Topology-Aware Generation

Finally, the article is generated section-by-section. By utilizing the optimized skeleton and hierarchical digests, the model ensures that every claim is grounded in the source material, maintaining coherence across the entire long-form document.

Experimental Validation: SurveyEval

To evaluate the framework, the researchers introduced SurveyEval, a benchmark consisting of 384 academic surveys in computer science paired with over 26,000 full-text reference papers. Unlike previous datasets that rely on abstracts, SurveyEval provides the complete ground truth required to measure actual resource utilization.

Key Results

  • Reference Utilization: LLM×MapReduce-V2 achieved a 95.5% Precision and 95.8% Recall on references, a massive improvement over the AutoSurvey baseline (50.1% / 51.7%).
  • Information Density: The framework generated the highest number of unique, informative claims, resulting in "meatier" articles with less redundancy.
  • Human Preference: In head-to-head human evaluations, LLM×MapReduce-V2 was preferred 62.7% of the time over AutoSurvey.
  • Optimization Efficiency: Testing revealed that skeleton refinement peaks at approximately 3 iterations; further iterations can lead to a drop in entropy as the model begins to deviate from the source material.

Conclusion and Future Work

LLM×MapReduce-V2 demonstrates that treating text synthesis as a convolutional integration problem significantly bridges the gap between vast input resources and coherent long-form output. While currently validated on academic surveys, future work will explore its applicability to other domains such as legal and financial reporting, as well as its performance across different reasoning-based models like DeepSeek-R1.

Summary updated

Abstract

Long-form generation is crucial for a wide range of practical applications, typically categorized into short-to-long and long-to-long generation. While short-to-long generations have received considerable attention, generating long texts from extremely long resources remains relatively underexplored. The primary challenge in long-to-long generation lies in effectively integrating and analyzing relevant information from extensive inputs, which remains difficult for current large language models (LLMs). In this paper, we propose LLM$\times$MapReduce-V2, a novel test-time scaling strategy designed to enhance the ability of LLMs to process extremely long inputs. Drawing inspiration from convolutional neural networks, which iteratively integrate local features into higher-level global representations, LLM$\times$MapReduce-V2 utilizes stacked convolutional scaling layers to progressively expand the understanding of input materials. Both quantitative and qualitative experimental results demonstrate that our approach substantially enhances the ability of LLMs to process long inputs and generate coherent, informative long-form articles, outperforming several representative baselines. Both LLM$\times$MapReduce-V2 and SurveyEval are publicly available at https://github.com/thunlp/LLMxMapReduce .
BibTeX
@misc{wang2025llmmapreduce,
  author = {Haoyu Wang and Yujia Fu and Zhu Zhang and Shuo Wang and Zirui Ren and Xiaorong Wang and Zhili Li and Chaoqun He and Bo An and Zhiyuan Liu and Maosong Sun},
  title = {{LLM⨉MapReduce-V2: Entropy-Driven Convolutional Test-Time Scaling for Generating Long-Form Articles from Extremely Long Resources}},
  year = {2025},
  eprint = {2504.05732},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2504.05732},
}
Figure