Abstract
Text Simplification (TS) often lacks the nuance required to assist individuals with cognitive disabilities, a specialized domain known as Cognitive Simplification (CS). While general TS models benefit from large parallel corpora, CS suffers from a significant data bottleneck. This paper proposes a novel method to bridge this gap by incorporating CS operations—such as rephrasing, deletion, and explicitation—as an inductive bias in neural models. By training models to predict these specific operations alongside the simplified text, we demonstrate significant performance gains on standard TS benchmarks and superior adaptation to the data-scarce CS domain.
Introduction
Text Simplification (TS) is a critical NLP task aimed at making information more accessible. However, most current research treats TS as a "black box" translation task, mapping complex sentences to simpler ones based on general-purpose datasets like Wikipedia. This approach often fails to meet the specific needs of readers with cognitive disabilities (e.g., dementia, developmental disabilities, or head trauma).
Cognitive Simplification (CS) follows professional manuals to ensure accessibility, but the field is hampered by a lack of large-scale parallel training data. This research addresses two primary challenges: leveraging the structured rules of CS to improve general TS models and enhancing model performance on CS tasks despite limited training data.
Methodology: Operations as Inductive Bias
The core of the proposed approach is a multi-task learning framework where the model learns not only to simplify text but also to identify the specific operations required for that simplification.
Operation Mapping
The authors distilled professional CS manuals into seven fundamental operations:
- Proximation: Making text more relatable to the reader.
- Rephrasing: Swapping complex terms for simpler synonyms.
- Deleting Information: Removing non-essential or confusing details.
- Adding Information: Providing necessary context.
- Explicitation: Making implicit information explicit.
- Intra-Sentence Rearrangement: Changing word order for clarity.
- Sentence-Level Operations: Splitting or merging sentences.
The "+Classifier" Model Architecture
Using T5 and BART architectures, the authors introduced an inductive bias via nine unique special tokens (e.g., <REPHRASE>, <DEL>, <PROX>). During the training phase on standard TS data (WikiAuto), the researchers used deterministic heuristics—utilizing parse trees and paraphrase databases—to "silver-label" the dataset with these operation tokens. At inference, the model is required to predict the appropriate operation tokens before generating the simplified output string.
Experimental Setup and Datasets
The models were evaluated across both general and specialized datasets:
- WikiAuto & ASSET: Standard benchmarks for general text simplification.
- FestAbility Transcripts: A novel test set introduced in this paper, consisting of 321 sentence pairs from a virtual accessibility conference, simplified live by professional cognitive simplifiers.
- Manual Dataset: A high-quality training set derived from manually aligned portions of WikiManual and Newsela.
To validate the distinction between general TS and CS, the authors performed a Jensen-Shannon Distance (JSD) analysis. The results confirmed that the FestAbility dataset is statistically distinct, characterized by a higher frequency of reordering and explicitation compared to standard TS corpora.
Results and Discussion
The inclusion of operation tokens consistently improved performance across all metrics.
General Text Simplification
On the WikiAuto/ASSET benchmark, the T5Base+Classifier model achieved a SARI score of 38.13, a significant improvement over the vanilla baseline of 32.01 (+6.12 points). This suggests that even for general tasks, explicit operation modeling helps the neural architecture better understand the transformation process.
Cognitive Simplification and Domain Adaptation
The models showed remarkable success on the new FestAbility dataset. The BART-Large+Classifier reached a SARI score of 27.13, outperforming the baseline significantly. Notably, the models demonstrated strong zero-shot adaptation; even when trained only on general TS data, the operation-based inductive bias allowed them to perform better on CS tasks than standard models. This indicates that the operations serve as a "universal language" of simplification that transcends specific datasets.
Limitations and Future Work
While the results are promising, the authors acknowledge that the heuristic labeling process is noisy, particularly for the <ADD> operation, which showed lower precision. Additionally, computational constraints limited the study to T5 and BART architectures, leaving the potential of larger models like GPT-4 or T5-11B for future exploration. Future research will focus on refining operation identification and expanding the CS framework to multi-lingual contexts.
Conclusion
By integrating cognitive simplification operations into neural models, this research provides a pathway to more accessible text. The use of operation tokens as an inductive bias not only improves performance on general benchmarks but also offers a robust solution for domain adaptation in data-scarce environments like cognitive accessibility.
Summary updated