DYPLOC: Dynamic Planning of Content Using Mixed Language Models for Text Generation

Xinyu Hua, Ashwin Sreevatsa, Lu Wang

Abstract

We study the task of long-form opinion text generation, which faces at least two distinct challenges. First, existing neural generation models fall short of coherence, thus requiring efficient content planning. Second, diverse types of information are needed to guide the generator to cover both subjective and objective content. To this end, we propose DYPLOC, a generation framework that conducts dynamic planning of content while generating the output based on a novel design of mixed language models. To enrich the generation with diverse content, we further propose to use large pre-trained models to predict relevant concepts and to generate claims. We experiment with two challenging tasks on newly collected datasets: (1) argument generation with Reddit ChangeMyView, and (2) writing articles using New York Times' Opinion section. Automatic evaluation shows that our model significantly outperforms competitive comparisons. Human judges further confirm that our generations are more coherent with richer content.

1. Introduction

Generating long-form opinion text—such as persuasive arguments or editorial articles—remains a significant challenge for neural language models. While large-scale pre-trained models exhibit high local fluency, they frequently struggle with global coherence and systematic content planning. Existing approaches typically rely on static outlines, which generators may ignore, or implicit attention mechanisms that lack fine-grained control over complex reasoning. Furthermore, opinion writing requires a delicate balance of objective facts (entities and concepts) and subjective arguments (claims), a synthesis that standard models often fail to achieve effectively.

2. Methodology

To address these challenges, we introduce DYPLOC (Dynamic Planning of Content), a framework that interleaves content planning with the generation process using a novel mixed language model architecture.

2.1 Content Item Augmentation

Before generation begins, DYPLOC enriches sparse inputs through two primary augmentation steps:

  • Concept Expansion: A fine-tuned BART model predicts specific, relevant concepts based on the input title to provide semantic anchors.
  • Claim Generation: The system generates "draft claims"—full sentences representing central propositions—to establish a structural and stylistic backbone for the argument.

2.2 Mixed Language Models and Dynamic Planning

Instead of processing all information as a single concatenated sequence, DYPLOC treats each "content item" (comprising entities, concepts, and a claim) as an independent input.

Overview of the DYPLOC framework, illustrating the parallel encoding of augmented content items and the dynamic plan scoring mechanism.

As shown in the architecture, each item is encoded in parallel. During decoding, a "plan scoring network" evaluates the generated context at every time step to assign weights to each content item. The final probability of the next token is a weighted sum of predictions from all item-conditioned models, allowing the generator to shift focus dynamically across the planned content items as the text progresses.

3. Experimental Setup

We evaluated DYPLOC on two challenging datasets:

  • Reddit ChangeMyView (CMV): Focused on generating counter-arguments to subjective opinions.
  • New York Times (NYT) Opinion: Focused on writing fact-dense editorial articles.

4. Results and Discussion

DYPLOC significantly outperformed competitive baselines, including standard sequence-to-sequence (SEQ2SEQ) models and BART models with concatenated augmented data (SEQ2SEQFULL). On the NYT dataset, DYPLOC achieved a BLEU-2 score of 40.63, a substantial improvement over the SEQ2SEQFULL baseline of 31.06.

Human evaluation results comparing DYPLOC against standard sequence-to-sequence baselines across multiple dimensions.

Human evaluation (Table 6) confirms these findings. Professional judges consistently rated DYPLOC higher in grammar, coherence, relevance, and content richness. Notably, DYPLOC was selected as the "best" output (Top-1) in 44.7% of cases for CMV and 45.9% for NYT. Ablation studies further highlighted that concepts are the most critical element for performance, acting as essential "anchor points" for coherent long-form generation.

5. Conclusion and Future Work

DYPLOC demonstrates that dynamic planning via mixed language models effectively bridges the gap between raw data and coherent, persuasive text. While the model represents a significant step forward, future work will address current limitations such as stance consistency and the requirement for "silver labels" during training. Automating the selection of content items for claim generation remains a logical next step for the framework.

Summary updated

Abstract

We study the task of long-form opinion text generation, which faces at least two distinct challenges. First, existing neural generation models fall short of coherence, thus requiring efficient content planning. Second, diverse types of information are needed to guide the generator to cover both subjective and objective content. To this end, we propose DYPLOC, a generation framework that conducts dynamic planning of content while generating the output based on a novel design of mixed language models. To enrich the generation with diverse content, we further propose to use large pre-trained models to predict relevant concepts and to generate claims. We experiment with two challenging tasks on newly collected datasets: (1) argument generation with Reddit ChangeMyView, and (2) writing articles using New York Times' Opinion section. Automatic evaluation shows that our model significantly outperforms competitive comparisons. Human judges further confirm that our generations are more coherent with richer content.
BibTeX
@inproceedings{hua2021dyploc,
  author = {Xinyu Hua and Ashwin Sreevatsa and Lu Wang},
  title = {{DYPLOC: Dynamic Planning of Content Using Mixed Language Models for Text Generation}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2021},
  url = {https://openreview.net/forum?id=VEG-T3hYgML},
}
Figure