Abstract
Long-form generation is crucial for a wide range of practical applications, typically categorized into short-to-long and long-to-long generation. While short-to-long generations have received considerable attention, generating long texts from extremely long resources remains relatively underexplored. The primary challenge in long-to-long generation lies in effectively integrating and analyzing relevant information from extensive inputs, which remains difficult for current large language models (LLMs). In this paper, we propose LLM$ imes$MapReduce-V2, a novel test-time scaling strategy designed to enhance the ability of LLMs to process extremely long inputs. Drawing inspiration from convolutional neural networks, which iteratively integrate local features into higher-level global representations, LLM$ imes$MapReduce-V2 utilizes stacked convolutional scaling layers to progressively expand the understanding of input materials. Both quantitative and qualitative experimental results demonstrate that our approach substantially enhances the ability of LLMs to process long inputs and generate coherent, informative long-form articles, outperforming several representative baselines.
Introduction: Beyond Extractive Retrieval
In the era of long-context Large Language Models (LLMs), the challenge of "long-to-long" generation—such as creating a 10,000-word academic survey from thousands of pages of research—remains a significant hurdle. Current approaches often rely on Retrieval-Augmented Generation (RAG), which primarily functions as an extractive process. RAG identifies specific snippets that match a query but frequently overlooks nuanced connections and broader context hidden in the "non-matching" portions of the text.
When faced with extremely long inputs, LLMs often suffer from the "lost in the middle" phenomenon or fail to synthesize a cohesive narrative. LLM×MapReduce-V2 addresses these bottlenecks by shifting from an extractive paradigm to an integrative one, treating massive document collections as a hierarchical structure to be synthesized rather than a database to be queried.
The LLM×MapReduce-V2 Framework
The framework introduces a "test-time scaling" strategy inspired by Convolutional Neural Networks (CNNs). Just as a CNN aggregates local pixels into textures and then into objects, LLM×MapReduce-V2 aggregates local information into higher-level global representations through three main stages:
1. Initialization
The process begins by creating a "Survey Tree" (a skeleton or outline) and "Digests"—compressed summaries of input papers tailored specifically to that skeleton. This ensures that the initial data reduction is guided by the desired final structure.
2. Entropy-Driven Convolutional Scaling
This stage is the core innovation. The framework uses stacked convolutional layers to iteratively integrate local feedback into global representations.
- Randomized Convolutional Layers: Instead of attempting to process all data simultaneously, the system samples sets of feedback items.
- Entropy-Driven Sampling: To optimize quality, the system uses an "LLM-as-judge" to estimate the information entropy (informativeness) of different skeleton versions. It prioritizes paths that maximize this entropy, ensuring the final representation is as informative as possible.
3. Topology-Aware Generation
Finally, the article is generated section-by-section. By utilizing the optimized skeleton and hierarchical digests, the model ensures that every claim is grounded in the source material, maintaining coherence across the entire long-form document.
Experimental Validation: SurveyEval
To evaluate the framework, the researchers introduced SurveyEval, a benchmark consisting of 384 academic surveys in computer science paired with over 26,000 full-text reference papers. Unlike previous datasets that rely on abstracts, SurveyEval provides the complete ground truth required to measure actual resource utilization.
Key Results
- Reference Utilization: LLM×MapReduce-V2 achieved a 95.5% Precision and 95.8% Recall on references, a massive improvement over the AutoSurvey baseline (50.1% / 51.7%).
- Information Density: The framework generated the highest number of unique, informative claims, resulting in "meatier" articles with less redundancy.
- Human Preference: In head-to-head human evaluations, LLM×MapReduce-V2 was preferred 62.7% of the time over AutoSurvey.
- Optimization Efficiency: Testing revealed that skeleton refinement peaks at approximately 3 iterations; further iterations can lead to a drop in entropy as the model begins to deviate from the source material.
Conclusion and Future Work
LLM×MapReduce-V2 demonstrates that treating text synthesis as a convolutional integration problem significantly bridges the gap between vast input resources and coherent long-form output. While currently validated on academic surveys, future work will explore its applicability to other domains such as legal and financial reporting, as well as its performance across different reasoning-based models like DeepSeek-R1.
Summary updated