Reformulating Discourse Connectives for Non-Expert Readers

Advaith Siddharthan, Napoleon KatsosNAACL 2010

TL;DR

This paper investigates how different ways of expressing causation in scientific text affect reader comprehension, particularly for non-expert audiences. They developed a machine learning model that can predict which phrasing is most acceptable, demonstrating that tailoring language can significantly improve readability for lay readers.

What Problem Does This Paper Address?

Scientific writing often uses complex language and specific phrasing that can be a barrier for non-expert readers. While discourse connectives like "because" and "cause" are crucial for conveying relationships, their syntactic and lexical variations can significantly impact how easily a reader understands the text. This is especially true for causal relationships, which are fundamental to scientific explanations. The authors aim to bridge this gap by understanding how different formulations of causation are perceived by readers with varying levels of domain expertise and by developing a system to automatically select the most appropriate phrasing for a given audience. This is particularly relevant for tasks like summarizing scientific literature for the public.

Methods

The researchers employed a multi-pronged approach:

  • Psycholinguistic Experiment: They conducted a study where participants rated the acceptability of various reformulations of causal sentences. This involved creating a dataset of 144 original sentences, each reformulated in 8 different ways, resulting in 1152 sentences in total. Participants used a magnitude estimation paradigm to provide nuanced acceptability judgments.
  • Feature Engineering: To train a machine learning model, they extracted shallow linguistic features from the sentences, including the type of discourse connective used, the genre of the text (PubMed abstracts, BNC World, BNC Natural Science), and various complexity measures (sentence length, clause length ratios, presence of copula, quantifiers).
  • Machine Learning Classification: They trained a classifier (specifically, a J48 decision tree) to predict sentence acceptability. A novel aspect of their approach was a two-stage classification process: the first run predicted acceptability based on surface features, and the second run incorporated these predictions as an additional feature, allowing the model to learn relative probabilities of acceptability for different formulations of the same propositional content.
  • Selective Reformulation: Finally, they integrated the machine learning predictions with user preferences (specifically, those of non-science students) to develop a "selective reformulation" strategy. This strategy aims to replace dispreferred formulations with more acceptable ones, guided by both general audience preferences and sentence-specific acceptability predictions.

Datasets

  • Custom Corpus: The authors created a new dataset of 144 original sentences, each reformulated in 8 ways, totaling 1152 sentences. These sentences were drawn from three genres: PubMed Abstracts (technical biomedical writing), BNC World (general news), and BNC Natural Science (popular science writing). This dataset is a key contribution, providing a benchmark for studying the acceptability of different discourse connective formulations.
  • Existing Corpora (for source material): PubMed (for abstracts), British National Corpus (BNC) World and Natural Science sections.

Key Insights and Ideas

  • Audience Matters: There are significant differences in how science and non-science students perceive the acceptability of different causal formulations. Non-science students, in particular, show stronger preferences and dispreferences.
  • Genre Influences Preference: While scientists might prefer technical writing (like PubMed abstracts) due to familiarity, both groups tended to disprefer popular science writing, suggesting that reformulating for lay audiences is challenging.
  • Surface Features Predict Acceptability: Linguistic features related to sentence complexity and the specific connective used are strong indicators of acceptability, even for readers unfamiliar with the domain.
  • Blind Reformulation is Counterproductive: Simply replacing dispreferred phrases with generally preferred ones can actually decrease overall acceptability, highlighting the need for context-aware reformulation.
  • Two-Stage Classification Improves Performance: Incorporating relative acceptability scores from an initial classification run significantly boosts the accuracy of the final model.

Key Results

  • A machine learning classifier achieved high accuracy (.89) and agreement (Kappa = .73) in predicting sentence acceptability for PubMed genre sentences.
  • The "selective reformulation" strategy, which combines general audience preferences with machine predictions, improved the average acceptability score of PubMed sentences for non-science students from 0.613 to 0.713. This brought the readability level to that comparable to their preferred World News genre.
  • Non-science students strongly dispreferred passive voice constructions and the use of "cause" as a noun, while preferring "because of" over "because" in certain contexts. Science students showed fewer strong preferences, likely due to familiarity and domain expertise.

Limitations and Future Work

  • Scope of Reformulation: The current reformulation method is limited to generating specific types of paraphrases (e.g., focusing on "cause" and "because" variants) and does not cover all possible syntactic alterations.
  • Subjectivity in Manual Reformulation: While efforts were made to standardize the process, some subjectivity in manual reformulation remains.
  • Future Work: The authors plan to extend their framework to other discourse relations commonly found in scientific writing (e.g., contrast). They also aim to develop a more comprehensive system using transfer rules and a bi-directional grammar to automate complex syntactic reformulations, further enhancing the ability to tailor scientific text for diverse audiences.

Summary updated

BibTeX
@inproceedings{siddharthan2010reformulating,
  author = {Advaith Siddharthan and Napoleon Katsos},
  title = {{Reformulating Discourse Connectives for Non-Expert Readers}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2010},
}
Figure