TL;DR
This paper introduces a novel decoding strategy for language models that explicitly controls the "toxicity" of generated interpretations of out-of-context sentences. By mimicking how humans might interpret sentences with varying toxicity levels, the method aims to better understand and anticipate misunderstandings, especially in online communication.
What Problem Does This Paper Address?
Online communication, particularly on social media, often involves sentences taken out of their original context. This can lead to significant misunderstandings, as readers may interpret a sentence differently than the author intended, sometimes inferring unintended toxicity. While language models can generate interpretations, they often struggle to capture the nuanced and sometimes toxic ways humans might interpret ambiguous or decontextualized sentences. This paper tackles the challenge of generating diverse interpretations that not only align with the original sentence's meaning but also reflect the varying degrees of toxicity that humans might perceive, thereby helping to anticipate and explain potential misunderstandings.
Methods
The core contribution is a controlled decoding strategy that manipulates the output probabilities of a language model during the generation process, rather than altering the model architecture itself. This strategy is designed to be plug-and-play with existing language models. It enforces three key objectives, inspired by observed human interpretation patterns:
- Match Toxicity Level: The generated interpretations should have a toxicity level that aligns with the input sentence's toxicity. This ensures the tone is preserved.
- Gradually Relax Control as Toxicity Rises: For sentences with higher inherent toxicity, the model should allow for a broader range of interpretations (i.e., loosen toxicity constraints). This is based on the observation that human interpretations of more toxic sentences tend to have a wider spread in toxicity scores.
- Promote Diversity: The strategy encourages diversity in the toxicity levels across the set of generated interpretations for a single sentence. This is achieved through a heuristic rule that alternates the direction of toxicity adjustment for subsequent interpretations.
These objectives are implemented by calibrating the language model's softmax scores before the decoding algorithm (like nucleus sampling) is applied.
Datasets
The primary dataset used is OrigamIM (Allein and Moens, 2024; Allein et al., 2025). This dataset is specifically designed for interpretation modeling and contains 9,851 human-written interpretations for 2,018 sentences from Reddit posts. The authors fine-tuned and validated their models on the training and validation sets of OrigamIM and used the test set for evaluation.
Key Insights and Ideas
- Toxicity as a Control Knob: The paper highlights that toxicity isn't just something to be eliminated; it's a dimension of meaning that humans naturally perceive and that can be used as a control mechanism for generating more human-like interpretations.
- Contextual Toxicity Perception: Human interpretation of toxicity is subjective and context-dependent. The strategy acknowledges this by allowing for a wider range of interpretations for more toxic inputs, mirroring human behavior.
- Decoding-Time Control is Powerful: Modifying the output probabilities during decoding offers a flexible and effective way to control generation without needing to retrain or modify complex model architectures.
- Diversity in Interpretation is Key: Generating a single interpretation is insufficient. The ability to produce a set of interpretations with varying toxicity levels better captures the ambiguity and potential for misunderstanding inherent in out-of-context text.
Key Results
- Improved Alignment: The controlled decoding strategy consistently led to generated interpretations that better align with human-written interpretations in terms of both syntax (measured by METEOR) and semantics (measured by COMET) compared to base models without control.
- Semantic Improvement: COMET scores showed substantial increases, with improvements of 4.10% for BART, 5.54% for LLAMA, and 4.04% for T5 when all three objectives were enforced.
- Reduced Uncertainty: The strategy generally resulted in lower model uncertainty (perplexity) when generating interpretations.
- Toxicity Correlation: Correlation scores confirmed that the toxicity-controlled interpretations better captured the toxicity behavior observed in human interpretations.
- Variable Lambda Advantage: Using a variable weight (λ) for toxicity control, which adapts based on input sentence toxicity, yielded better results than a fixed λ.
- Content Moderation Potential: The paper notes that by adjusting the input toxicity score, the strategy can be used to generate non-toxic interpretations, which is valuable for content moderation.
Limitations and Future Work
- Classifier Dependency: The effectiveness of the toxicity control is directly tied to the quality of the external toxicity classifier used. If the classifier is biased or inaccurate, the control mechanism will be compromised.
- Beyond Toxicity: The paper acknowledges that human interpretation is influenced by many factors beyond toxicity, such as cultural background and personal experiences. Future work should aim to incorporate these broader contextual factors to model the diverse ways meaning is perceived.
Summary updated