Abstract
Recent approaches to data-to-text generation have adopted the very successful encoder-decoder architecture or variants thereof. These models generate text which is fluent (but often imprecise) and perform quite poorly at selecting appropriate content and ordering it coherently. To overcome some of these issues, we propose a neural model with a macro planning stage followed by a generation stage reminiscent of traditional methods which embrace separate modules for planning and surface realization. Macro plans represent high level organization of important content such as entities, events and their interactions; they are learnt from data and given as input to the generator. Extensive experiments on two data-to-text benchmarks (RotoWire and MLB) show that our approach outperforms competitive baselines in terms of automatic and human evaluation.
Introduction
Modern neural networks are remarkably adept at generating fluent text. However, in data-to-text generation tasks—such as converting sports box scores into news summaries—they frequently suffer from two critical flaws: hallucination and poor structural organization. While previous research has focused on "micro planning" (selecting individual data records), this work introduces a macro planning framework. By decoupling high-level content organization from surface realization, the model learns to "think before it speaks," resulting in more factual and coherent narratives.
Methodology
The proposed architecture mimics the traditional human writing pipeline by separating the task into two distinct stages: macro planning and generation.
Macro Planning Stage
The macro planner identifies "paragraph plans," which are clusters of entities (players/teams) and events (scoring plays) that naturally co-occur. Using a Pointer Network, the model selects which plans to include and determines their optimal sequence. This stage ensures that the document follows a logical narrative flow before any words are generated.
Generation Stage
The second stage is a standard encoder-decoder model equipped with attention and a copy mechanism. Crucially, instead of processing raw, unstructured data tables, it receives the structured macro plan. This allows the generator to focus exclusively on linguistic fluency and precise realization of the pre-selected content.
Datasets and Evaluation
The model was evaluated on two major benchmarks:
- ROTOWIRE: NBA basketball statistics and summaries (~5,000 instances).
- MLB: A larger, more complex dataset of Major League Baseball statistics (~25,000 instances) with summaries averaging over 500 words.
Key Results
The macro-planning approach significantly outperformed state-of-the-art baselines across several metrics:
- Factuality: On the MLB dataset, the model achieved a Relation Generation (RG) precision of 94.4%, the highest among neural models, indicating a substantial reduction in hallucinations.
- Content Selection: The model saw a significant jump in Content Selection (CS) F-measure on ROTOWIRE (42.9 vs. 40.5), demonstrating superior ability in identifying relevant information.
- Human Evaluation: Human raters consistently scored the Macro model higher for coherence and grammaticality compared to end-to-end neural baselines.
Conclusion and Future Work
By treating paragraphs as the primary "unit of thought" and decoupling logic from language, this framework provides a robust solution for long-form data-to-text generation. Future research will address remaining challenges, such as "attention diffusion" in extremely long documents and improving the model's numerical reasoning capabilities to prevent the misinterpretation of statistical data.
Summary updated