Learning to Substitute Words with Model-based Score Ranking

Hongye Liu, Ricardo Henao

Introduction

Smart Word Substitution (SWS) aims to enhance text quality by suggesting contextually appropriate word alternatives. Unlike traditional lexical substitution, which focuses on semantic equivalence, SWS seeks to improve the "naturalness" or impact of a sentence. However, developing robust SWS systems is hindered by the inherent subjectivity of word choice. Human annotators often disagree on optimal substitutions, making ground-truth labels sparse and expensive to obtain. This paper proposes a novel framework that bypasses human labeling by using automated quality metrics as a proxy for training.

The Challenge of Subjectivity

A core motivation for this work is the observation that word substitution is rarely a binary "right or wrong" task. Analysis of the SWS benchmark reveals that in 94% of cases, both humans and models prefer to leave a token unchanged. When a change is suggested, the agreement between different annotators or between humans and models is remarkably low (approximately 11%).

Table 1: Statistics on word change and agreement, highlighting the sparse and subjective nature of human-labeled data.

This high level of disagreement suggests that training models to predict a single "correct" human label is suboptimal. Instead, the task is better framed as learning the relative quality of various candidates.

Methodology: Ranking with Model-based Scores

The researchers introduce a self-supervised approach that treats word substitution as a ranking problem guided by BARTScore, a metric that estimates sentence quality based on the log-likelihood of a pre-trained BART model.

1. BARTScore as a Teacher

BARTScore serves as the objective function. By calculating the score for a sentence before and after a substitution, the system can determine if a specific word choice improves grammatical correctness or semantic flow.

2. Margin Ranking (MR) Loss

Rather than performing classification, the model is trained using a Margin Ranking loss. For a set of $K$ candidate substitutes, the model learns to assign higher probabilities to words that result in higher BARTScores. This ensures the model understands the relative hierarchy of word quality.

3. Average Score (AS) Improvement

To prevent the model from simply ranking mediocre options, an Average Score (AS) improvement term ($L_{AS}$) is added. This term encourages the model to maximize the weighted average quality score across its top-ranked predictions, pushing the model toward high-quality suggestions.

The final model, MR+AS, is built upon a fine-tuned BERT-base-uncased architecture.

Experimental Results

The MR+AS model was evaluated against several baselines, including vanilla masked language models (BERT, BART) and state-of-the-art Large Language Models (GPT-4o, LLaMA-3.1), across four datasets: SWS, LS07, LS14, and XSum.

Comparative Performance

The primary metrics for evaluation were Cosine Similarity (CS), measuring the alignment between model likelihoods and BARTScores, and the Average BARTScore Ratio (ABR), measuring the quality of the top suggestions.

Table 2: Performance comparison across datasets. MR+AS consistently achieves the highest alignment (CS) and competitive quality (ABR) compared to LLMs.

As shown in the results, MR+AS achieved a median CS of 0.99, significantly outperforming vanilla BERT (0.92) and even LLaMA (0.94). In terms of quality (ABR), the specialized MR+AS model (0.86) matched or exceeded the performance of GPT-4o (0.86) across the board. This indicates that for high-precision tasks like word substitution, a well-tuned smaller model can outperform much larger general-purpose LLMs.

Human Validation

A human preference study further validated these findings. When the MR+AS model and BARTScore agreed on a ranking, human annotators favored the suggested choice 79% of the time. Even in cases of disagreement, humans sided with the BARTScore-preferred choice 67% of the time, confirming that model-based scores are effective proxies for human judgment.

Conclusion

This research demonstrates that model-based quality metrics like BARTScore have matured sufficiently to serve as "label proxies" for training specialized language models. By framing SWS as a ranking problem and optimizing for relative quality rather than absolute labels, the MR+AS framework achieves state-of-the-art performance while eliminating the need for costly human annotation. Future work will explore the integration of other scoring metrics and methods to increase the substitution rate without sacrificing quality.

Summary updated

Abstract

Smart word substitution aims to enhance sentence quality by improving word choices; however current benchmarks rely on human-labeled data. Since word choices are inherently subjective, ground-truth word substitutions generated by a small group of annotators are often incomplete and likely not generalizable. To circumvent this issue, we instead employ a model-based score (BARTScore) to quantify sentence quality, thus forgoing the need for human annotations. Specifically, we use this score to define a distribution for each word substitution, allowing one to test whether a substitution is statistically superior relative to others. In addition, we propose a loss function that directly optimizes the alignment between model predictions and sentence scores, while also enhancing the overall quality score of a substitution. Crucially, model learning no longer requires human labels, thus avoiding the cost of annotation while maintaining the quality of the text modified with substitutions. Experimental results show that the proposed approach outperforms both masked language models (BERT, BART) and large language models (GPT-4, LLaMA). The source code is available at https://github.com/Hyfred/Substitute-Words-with-Ranking.
BibTeX
@inproceedings{liu2025learning,
  author = {Hongye Liu and Ricardo Henao},
  title = {{Learning to Substitute Words with Model-based Score Ranking}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2025},
  doi = {10.18653/v1/2025.naacl-long.576},
}
Figure