Introduction
While Large Language Models (LLMs) have made significant strides in general alignment, they often lack the nuanced "social intelligence" required to navigate specific cultural contexts. A behavior considered standard in one culture may constitute a "norm violation" in another, potentially leading to social friction. Current research predominantly focuses on detecting these violations or aligning models with broad universal values. However, there remains a critical gap in remediation: the ability of an AI to assist users in rephrasing offensive statements or understanding underlying social rules. This paper introduces RENOVI, a benchmark focused on remediating norm violations within Chinese culture, where hierarchy, politeness markers, and indirectness play pivotal roles.
Methodology: The Remediation Pipeline
The researchers operationalized Expectancy Violation Theory (EVT) and Interaction Adaptation Theory (IAT) to create a structured four-step pipeline for handling social faux pas.
The pipeline consists of:
- Norm Violation Detection: Identifying if an utterance breaks a social norm and categorizing the interaction (e.g., a request).
- Impact Estimation: Determining the severity of the violation (High vs. Low Impact).
- Remediation Generation: Rephrasing the violating utterance to align with cultural expectations.
- Justification Generation: Explaining the rationale behind the fix to educate the user on the relevant social rule.
The RENOVI Dataset
The primary contribution is the RENOVI Dataset, comprising 9,258 multi-turn dialogues. To ensure both quality and scale, the authors employed a hybrid approach: 512 dialogues were meticulously authored by humans, while 8,746 were synthetically generated using ChatGPT with ontology-based prompts.
The dataset covers seven core social norms: apology, criticism, greeting, persuasion, request, leave-taking, and thanking. Each entry is annotated with situational attributes such as location and role relationships (e.g., boss/subordinate).
Key Insights and Results
The Synthetic Data Trap
A significant finding was that training models exclusively on synthetic data can degrade performance due to "domain shift." Synthetic dialogues tend to be more clustered and less diverse than human-authored ones. However, using a small amount of human data as an "anchor" alongside synthetic data significantly boosts model performance, suggesting that synthetic data acts as an effective amplifier when grounded in human examples.
Model Performance
- Detection & Impact: Fine-tuned models like RoBERTa-zh achieved F1 scores of approximately 60% for detection and 78% for impact estimation when trained on the mixed dataset.
- Remediation: ChatYuan, a Chinese-specific LLM, combined with a Pfeiffer adapter, proved most effective at producing concise remediations that matched human style and length.
- Human Evaluation: Human judges rated LLM-generated justifications highly for relevance and informativeness, though they noted that models still occasionally struggle with the nuances of politeness in rephrasing.
Conclusion and Future Work
RENOVI provides a structured framework for AI to learn, correct, and explain social errors. While currently focused on Chinese culture, the authors intend to expand the benchmark to include Spanish, Latin, and Arabic contexts. By moving beyond simple detection toward active remediation, RENOVI paves the way for interactive systems that serve as true socio-cultural bridges in global communication.
Summary updated