When science journalism meets artificial intelligence : An interactive demonstration

Raghuram Vadapalli, Bakhtiyar Syed, Nishant Prabhu, Balaji Vasan Srinivasan, Vasudeva VarmaEMNLP 2018

TL;DR

This paper introduces an interactive tool that automatically generates engaging blog titles from scientific paper titles and abstracts, aiming to bridge the gap between complex research and public understanding. By creating a novel parallel corpus of scientific articles and their corresponding blog posts, the authors demonstrate a two-stage pipeline that combines heuristic information extraction with a pointer-generator neural network for semantically sound and readable title generation.

What Problem Does This Paper Address?

The sheer volume of scientific publications (2.5 million new papers annually) makes it challenging to disseminate crucial research findings to the general public. While science journalism plays a vital role in translating jargon-laden articles into accessible content, the process is labor-intensive. Existing "robo-reporting" has shown promise in automating news generation, but extending this to the nuanced domain of science journalism, which requires deep understanding and accurate simplification, has been hindered by a lack of suitable training data. This paper addresses this gap by creating a new dataset and developing a system to automate the first step of science journalism: generating compelling blog titles.

Methods

The proposed system employs a two-stage pipeline:

  1. Heuristic Information Extraction (Stage 1): This stage takes the research paper's title and abstract as input and uses various heuristic functions to extract relevant information. These functions include selecting the paper title itself, identifying the most representative sentence in the abstract using TF-IDF (RP(abs)), finding the most readable sentence (RD(abs)), or a combination of these. The output is an intermediate sequence of extracted information.
  2. Sequence-to-Sequence Generation (Stage 2): The intermediate sequence from Stage 1 is fed into a neural generation model. The authors specifically utilize a pointer-generator network. This model, an extension of attention-based sequence-to-sequence architectures, can both generate new words from a vocabulary and copy words directly from the input sequence. This "copying" capability is hypothesized to be crucial for retaining factual information from the source material when generating blog titles.

Datasets

The authors introduce a novel parallel corpus specifically for science journalism. This corpus consists of 87,328 triples of (blog title, research paper title, research paper abstract). This dataset was constructed by aggregating data from two science news aggregators: sciencedaily.com and phys.org. While over 300,000 blog titles were initially collected, only about 100,000 had links to original research papers, from which titles and abstracts were then extracted.

Key Insights and Ideas

  • Bridging the Science-Journalism Divide: The core idea is to leverage AI to automate the initial, crucial step of making scientific research accessible to a wider audience by generating engaging blog titles.
  • The Power of the Pointer-Generator: The pointer-generator network is highlighted as a key component. Its ability to copy words directly from the source text is essential for preserving the factual essence of scientific content, which is critical for accurate science communication.
  • Heuristics as a Preprocessing Step: Using heuristic functions to pre-process and extract salient information from research papers before feeding it to the neural model helps to focus the generation process and improve relevance.
  • Importance of Readability: The generated blog titles aim for higher readability (Flesch Reading Ease scores) compared to the original research paper titles, indicating a successful style transfer towards a more accessible format.

Key Results

  • The system successfully generates blog titles that are semantically sound and exhibit a transfer in style towards greater readability.
  • The generated blog titles have a higher Flesch Reading Ease (FRE) score (around 30-35) compared to research paper titles (15-20), making them easier for a general audience to understand.
  • The paper presents a comprehensive evaluation of various heuristic functions and generation models using metrics like BLEU, ROUGE-L, CIDEr, and Skip-Thought Cosine Similarity, demonstrating the viability of the proposed system.
  • Qualitative analysis of sampled generated titles shows they are similar to those produced by human experts for scientific blogs.

Limitations and Future Work

  • Nascent Stage: The authors acknowledge that their prototype is in its early stages, and the problem of automated science journalism is far from fully solved.
  • Input Specificity: A single heuristic function or neural generation model may not perform optimally for all types of scientific content.
  • Future Work: The immediate future work involves using more advanced architectures to generate the full body of a blog post, moving beyond just titles. The system's flexibility in allowing users to select heuristic functions and neural models is also emphasized as a way to achieve better results for specific use-cases.

Summary updated

BibTeX
@inproceedings{vadapalli2018when,
  author = {Raghuram Vadapalli and Bakhtiyar Syed and Nishant Prabhu and Balaji Vasan Srinivasan and Vasudeva Varma},
  title = {{When science journalism meets artificial intelligence : An interactive demonstration}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2018},
}
Figure