Writing Strategies for Science Communication: Data and Computational Analysis

Tal August, Lauren Kim, Katharina Reinecke, Noah A. SmithEMNLP 2020 · OpenReview

Introduction

Science communication (SciComm) is often guided by prescriptive advice—"avoid jargon," "tell a story," or "lead with the punchline." While these tips are intuitive, they are rarely backed by large-scale empirical evidence. This research bridges the gap between SciComm theory and practice by developing a computational framework to identify and analyze ten core writing strategies across a massive dataset of science news.

By formalizing these strategies into a computational task, the authors aim to understand how writing styles shift across different audiences and whether specific narrative techniques correlate with higher writing quality.

Methodology

The researchers developed a multi-stage Natural Language Processing (NLP) pipeline to classify writing strategies:

  1. Strategy Definition: Ten strategies were distilled from professional style guides and expert interviews, including LEDE (the hook), MAIN (the core finding), IMPACT (real-world significance), ANALOGY, and STORY (narrative elements).
  2. Detection Framework:
    • Rule-Based: Strategies like Jargon, Active Voice, and Present Tense were identified using linguistic rules and specialized lexicons.
  3. Transformer-Based: For nuanced strategies like storytelling, the authors fine-tuned RoBERTa models using domain-adaptive pretraining on 11.9 million news articles and task-adaptive pretraining on a specialized science corpus.
  4. Calibration: To account for the subjectivity of discourse labels, the team used "Probabilistic Classify and Count" (PCC) to ensure accurate estimates of strategy prevalence across large datasets.

Data and Venue Analysis

The study introduced a corpus of 128,253 science writing documents published between 2016 and 2019. This dataset spans four distinct "venues," each catering to different audiences.

Table 1: Distribution of the 128,253 articles across various SciComm venues and specific sites.

Analysis revealed that these venues are linguistically distinct. Using Principal Component Analysis (PCA), the authors found that publication types cluster based on strategy usage. For instance, popular magazines function as "storytellers," while blogs act as "technical reporters," often retaining higher levels of jargon to serve science-literate readers.

Key Insights

The Role of Narrative and Impact

One of the most striking findings involves the use of narrative. When comparing "Typical" science articles to "Great" ones (those featured in Best American Science Writing anthologies), the higher-quality pieces used significantly more Storytelling and Analogies.

Conversely, university press releases were found to use a disproportionately high number of IMPACT sentences. This suggests that the primary goal of press releases is often to "sell" the importance of research to news desks rather than purely explaining the science.

Figure 2: Distribution of strategy usage across different sites, highlighting how venues like The Atlantic (STORY) differ from technical blogs.

Model Performance and Strategy Prevalence

The RoBERTa classifiers achieved $F_1$ scores between 0.40 and 0.55. While seemingly modest, these results are consistent with other complex discourse-tagging tasks. Across the entire corpus, "Storytelling" and "Main Findings" were the most prevalent strategies, while "Analogies" remained the rarest.

Conclusion and Future Directions

This study provides a data-driven foundation for SciComm, yet it also highlights the challenges of automating discourse analysis. Strategies like "Explanation" and "Personal Details" proved difficult to model due to low inter-annotator agreement, reflecting the inherent subjectivity of good writing.

Future work aims to move beyond sentence-level classification to document-level modeling and to integrate reader engagement metrics. Ultimately, this research paves the way for AI-powered writing assistants that can help scientists tailor their message to diverse public audiences effectively.

Summary updated

BibTeX
@inproceedings{august2020writing,
  author = {Tal August and Lauren Kim and Katharina Reinecke and Noah A. Smith},
  title = {{Writing Strategies for Science Communication: Data and Computational Analysis}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2020},
  url = {https://openreview.net/forum?id=xILgBQy0B1i},
}
Figure