Introduction
In writing education, a common technique to enhance prose is the addition of "engaging modifiers"—descriptive adjectives and adverbs that transform plain sentences into vivid imagery. For instance, "I ate a pear" becomes "I ate a big, juicy pear enjoyably." While effective, manually providing such feedback for every sentence in a student's essay is labor-intensive for educators.
Automating this task presents two primary technical hurdles. First, there is a significant data scarcity issue, as large-scale parallel datasets of "plain" versus "polished" sentences do not naturally exist. Second, models often fall into a copying trap; when tasked with editing a sentence that is already grammatically correct, they tend to repeat the input verbatim rather than generating creative enhancements.
Methodology
To address these challenges, we propose a self-supervised framework that leverages a large language model (LongLM) augmented with a retrieval strategy.
Automatic Dataset Construction
We employ a "perturbation-and-reconstruction" approach. By collecting approximately 50,000 engaging sentences from literature and 26,000 high-quality student essays, we use a natural language processing toolkit (LTP) to identify and strip away existing adjectives and adverbs. This process creates a synthetic parallel dataset where the model learns to reconstruct descriptive sentences from their stripped-down counterparts.
Retrieval-Augmented Generation
To ensure that added modifiers are semantically and contextually appropriate, we implement a retrieval-augmentation algorithm. During inference, the system queries a pre-constructed dictionary—derived from the training set—for suitable modifiers associated with the nouns and verbs in the source sentence. These candidates serve as prompts, guiding the model toward more accurate and evocative descriptions.
Sub-sentence Generation Strategy
To mitigate the copying trap, source sentences are partitioned into sub-sentences. During training, the model is penalized if it fails to generate the specific sub-sentences that require modification. This forces the model to focus on the creative task of insertion rather than simple reconstruction.
Experimental Setup
The dataset comprises roughly 160,000 parallel examples. While Train, Val, and TestAuto sets are automatically generated, we also include TestReal, a set of 1,000 human-written sentences that were not artificially corrupted. This allows us to evaluate the model's performance on natural, unstripped text.
Results and Analysis
The proposed model was evaluated against several baselines, including a vanilla LongLM.
Quantitative Performance
On the TestReal set, our model achieved a significantly lower Copy Ratio (7.1%) compared to the vanilla LongLM (38.6%). This indicates that the model is far more proactive in suggesting meaningful edits rather than defaulting to the input text.
Qualitative Evaluation
Human annotators rated the model's outputs across three dimensions: Fluency, Correctness, and Engagingness. Our model scored high on Engagingness (3.44/5) and Fluency (1.88/2), demonstrating its ability to produce high-quality, attractive prose.
As shown in the examples, the model successfully introduces evocative phrases such as "greedily sucking" (贪婪地吮吸) to describe trees absorbing sunlight, whereas simpler models often fail to make any changes.
Conclusion
This work demonstrates that a self-supervised, retrieval-augmented approach can effectively automate the process of sentence polishing. By overcoming the copying trap and data scarcity, the system provides a valuable tool for writing assistance. Future work will explore more complex rhetorical devices, such as metaphors, and expand the system's capabilities to other languages beyond Chinese.
Summary updated