Controllable Sentence Simplification via Operation Classification

Liam Cripwell, Joël Legrand, Claire GardentNAACL 2022 · DOI

Abstract

Neural sentence simplification models often act as "black boxes," making it difficult to predict whether they will rephrase, split, or simply copy the input. This paper introduces a controllable pipeline that first classifies the required simplification operation—such as syntactic splitting or discourse-based splitting—and then uses that classification as a guide for a generative model. The result is a system that is more fluent, less conservative, and significantly more predictable than standard end-to-end approaches.

Introduction

Sentence simplification aims to make text easier to read while preserving its original meaning. While modern sequence-to-sequence neural models produce fluent text, they suffer from two major issues: conservatism and a lack of control. Conservatism arises because training data often contains many "identical" pairs, leading models to learn to make no changes. Furthermore, users cannot easily specify how a sentence should be simplified (e.g., shortening via rephrasing vs. breaking into two sentences). This paper argues for focusing on high-level operations to better mimic human simplification strategies.

Methods

The proposed system follows a two-step pipeline:

  1. Operation Classification: Four global operations are defined: Identity (no change), Rephrase (paraphrasing), Syntax-split (grammatical splitting), and Discourse-split (splitting based on discourse markers). A RoBERTa model is fine-tuned to predict these labels.
  2. Controllable Generation: A BART model performs the actual simplification, using the predicted operation label as a "control token" prepended to the input.

To train these models, the researchers created the IRSD dataset (Identity, Rephrase, Split-Syntax, Split-Discourse) by heuristically labeling and combining sources like Wiki-auto, Newsela-auto, WikiSplit, and D-CCNews.

Results

Classifier Performance

The RoBERTa classifier achieved exceptional performance on the silver-label test set, reaching 98% accuracy. This high accuracy demonstrates that the model effectively learns the heuristic patterns used to construct the IRSD dataset.

Confusion matrices for the operation classifier on the silver-label test set. Comparing (a) the 4-class model with (b) a 3-class model illustrates the 'Identity Trap,' where the absence of an 'Identity' class forces incorrect rephrasing.

However, performance on the human-annotated Gold Test Set was lower, reflecting the inherent subjectivity of the task. Humans often find multiple operations plausible for the same sentence, leading to lower inter-annotator agreement (Cohen’s Kappa indicating agreement only ~24.6% of the time).

Confusion matrices on the Gold Test Set. The 4-class model (a) shows lower diagonal values compared to silver data, highlighting the subjective nature of human simplification preferences and the difficulty of the task.

Generation Quality

The controllable model (Ctrl4,4) outperformed standard BART baselines and previous state-of-the-art models on SARI and BERTScore. Notably, the model showed a significant improvement in "Split Recall," indicating it successfully performs splits when prompted rather than defaulting to copying the input. In human evaluations, the model was rated highest for fluency (4.77/5.0) and overall quality.

Discussion and Limitations

A key insight from this work is the "Identity Trap": including identical complex-simple pairs in training data for rephrasing or splitting hurts performance. Treating "Identity" as a distinct class and excluding these noisy pairs from other classes significantly improved classifier accuracy. Furthermore, distinguishing between syntactic and discourse splits allows the model to better handle specific rewriting needs.

Despite these gains, limitations remain. The subjectivity of operation selection is a major challenge. Additionally, while the model excels at splitting, it sometimes fails to simplify vocabulary within the resulting sentences. Future work could explore "nested" operations, such as splitting followed by rephrasing.

Conclusion

By decomposing sentence simplification into operation classification and controlled generation, this work provides a more predictable and less conservative framework. The use of high-level operation tokens allows for better alignment with human-like editing strategies, paving the way for more interpretable and user-controllable text simplification systems.

Summary updated

BibTeX
@inproceedings{cripwell2022controllable,
  author = {Liam Cripwell and Joël Legrand and Claire Gardent},
  title = {{Controllable Sentence Simplification via Operation Classification}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2022},
  doi = {10.18653/v1/2022.findings-naacl.161},
}
Figure