Introduction
Neural text generation models, such as those used for machine translation and abstractive summarization, are typically trained using Maximum Likelihood Estimation (MLE) with "teacher forcing." While effective, this paradigm suffers from two primary issues: exposure bias and the objective mismatch. Exposure bias occurs because the model is trained on ground-truth tokens but must rely on its own potentially erroneous predictions during inference. The objective mismatch arises because models are optimized to predict the next word correctly (word-level) but are evaluated using sequence-level metrics like ROUGE or BLEU.
While Reinforcement Learning (RL) offers a path to optimize sequence-level rewards directly, it is notoriously unstable and suffers from high variance. This paper introduces CALCS (Continuously Approximating Longest Common Subsequence), a differentiable surrogate for the LCS metric that allows for direct sequence-level optimization via standard gradient-based methods.
The CALCS Method
The core innovation of CALCS is the "softening" of the Longest Common Subsequence (LCS) calculation. Standard LCS relies on a dynamic programming (DP) table where a discrete binary check (if y_i == z_j) determines the path. This step function is non-differentiable, preventing backpropagation.
CALCS replaces this hard check with the model's predicted probability $p_j^{(y_i)}$ of generating the ground-truth token $y_i$ at step $j$. To ensure the approximation remains tight, the authors employ a peaked softmax with a temperature hyperparameter ($\alpha$). As $\alpha$ approaches zero, the soft probability distribution converges to the true discrete LCS. Because the resulting DP recurrence consists only of differentiable operations (addition, multiplication, and max), the entire sequence-level score can be optimized directly.
The training process typically involves pre-training with standard MLE followed by fine-tuning with a joint loss function that incorporates both MLE and the CALCS objective.
Experimental Results
The authors evaluated CALCS on two major benchmarks: the CNN/Daily Mail dataset for summarization and the WMT 2014 English-to-German dataset for translation.
Summarization Performance
On the CNN/Daily Mail task, the addition of CALCS led to a substantial improvement in sequence similarity. Specifically, the ROUGE-L score increased by 2.2 points compared to the baseline model. The CALCS-enhanced model also achieved a ROUGE-1 score of 40.37, outperforming several established RL-based methods.
Translation Performance
In the WMT'14 English-to-German translation task, the model showed a modest but noticeable improvement of 0.2 BLEU points. Interestingly, although CALCS specifically approximates LCS (the foundation of ROUGE), it still yielded gains in BLEU, which is based on n-gram precision.
Discussion and Limitations
CALCS provides a stable alternative to RL for sequence-level optimization, integrating seamlessly into standard deep learning workflows without the need for complex variance-reduction techniques. By using the model's own soft predictions, it effectively mitigates exposure bias.
However, some challenges remain:
- Length Gaming: Models may attempt to generate excessively long sequences to maximize common subsequences. The authors addressed this by limiting decoding length, but more robust length penalties may be required.
- Computational Overhead: The 2D DP table required for CALCS adds computational complexity compared to standard word-level MLE.
- Metric Specificity: While CALCS targets LCS, future work could explore differentiable surrogates for other metrics like METEOR.
Overall, CALCS represents a principled step toward bridging the gap between training objectives and evaluation metrics in neural text generation.
Summary updated