TL;DR
This paper introduces NormGenesis, a novel framework for generating culturally appropriate dialogues across English, Chinese, and Korean. It tackles the challenge of modeling social norms in AI by introducing a "Violation-to-Resolution" (V2R) paradigm and an exemplar-guided refinement process, leading to more nuanced and socially competent conversational agents.
What Problem Does This Paper Address?
Current dialogue systems often struggle to generate responses that are not only coherent but also socially acceptable, especially across different cultures. This is particularly true for low-resource languages like Korean, where models frequently exhibit inconsistencies in politeness, emotional expression, and social dynamics. Existing datasets and methods often focus on English and lack the fine-grained annotations needed to capture the dynamic nature of social interactions, especially how individuals recover from norm violations. The paper aims to bridge this gap by creating a multicultural framework that models social norms, including how to repair norm violations, and generates culturally grounded dialogues.
Methods
NormGenesis employs a four-stage pipeline:
- Norm and Style Design: A taxonomy of 12 social norm categories is established, with 10 culturally grounded subnorms defined per language (English, Chinese, Korean). This stage leverages value indicators from surveys and LLM-guided alignment.
- Scenario-Situation Constructor: For each subnorm, scenario-situation pairs are generated, categorized as Norm Adherence, Norm Violation, or Violation-to-Resolution (V2R). V2R specifically models post-violation repair strategies.
- Exemplar-Based Iterative Refinement: This is a key innovation. Instead of post-hoc filtering, the framework uses manually curated exemplars to guide LLM-based refinement of scenario-situation pairs. This process iteratively improves linguistic quality and cultural alignment, especially crucial for low-resource languages.
- Multi-Turn Dialogue Generator with Turn-Level Annotation: Refined scenario-situation pairs are used to generate multi-turn dialogues (5-15 turns). Each utterance is then annotated with norm adherence, speaker reaction (intent, emotion), and justification, providing rich data for training and evaluation.
Datasets
The paper introduces NormGenesis, a new dataset comprising 10,800 multicultural, multi-turn dialogues across English, Chinese, and Korean. These dialogues are annotated at the turn level for norm adherence, speaker intent, and emotional response. The dataset includes three interaction types: Norm Adherence, Norm Violation, and Violation-to-Resolution (V2R).
For evaluation, the authors compare their approach against existing datasets like:
- NORMDIAL: A bilingual English-Chinese corpus for norm adherence and violation.
- SODA: An English corpus for socially appropriate dialogues.
- DAILYDIALOG (English) and LCCC (Chinese) are used for generalization testing.
Key Insights and Ideas
- Violation-to-Resolution (V2R) is Crucial: Modeling how speakers recover from norm violations is essential for realistic and socially competent dialogue. This goes beyond simply identifying violations.
- Exemplar-Guided Refinement is Powerful: Iterative refinement using expert-curated examples, applied before full dialogue generation, significantly improves cultural and pragmatic consistency, especially in low-resource languages where static prompts often fail.
- Cultural Nuance Matters: Direct translation or generic prompts are insufficient. Norms vary significantly across cultures (e.g., apologies, expressions of empathy), and models must capture these specificities.
- Low-Resource Languages Benefit Most: The exemplar-based refinement strategy is particularly effective in addressing the pragmatic failures often seen in models trained on or for low-resource languages.
Key Results
- NormGenesis dialogues consistently outperform baselines in norm alignment, emotional coherence, and repair quality, as confirmed by both human and automatic evaluations.
- Models trained on NormGenesis show improved pragmatic competence in ethically sensitive contexts.
- The V2R-augmented model was preferred in 82% of cases in a comparison on the PROSOCIALDIALOG dataset, highlighting the impact of modeling norm repair.
- Human evaluations showed strong agreement with LLM-based assessments, validating the automatic evaluation protocols.
- The refinement process, on average, required only 1.2 rounds per instance, demonstrating efficiency.
Limitations and Future Work
- Language Coverage: The framework currently supports English, Chinese, and Korean. Expanding to more typologically diverse languages (e.g., Arabic, Swahili, Hindi) is a key future direction.
- Exemplar Scalability: While more scalable than full annotation, managing a large number of exemplars can still be resource-intensive. Future work may explore active learning and exemplar repositories.
- Subjectivity and Bias: Evaluating social norms inherently involves subjectivity. While safeguards like native speaker review and cross-review were used, cultural bias remains a challenge.
- Norm Evolution: Social norms are dynamic. The framework's taxonomy is a snapshot, and future work could involve tracking norm shifts over time.
- Ethical Considerations: The dataset could be misused to train agents that generate harmful content. The authors emphasize responsible use for prosocial AI research.
- Generalizability: While tested on Malay and Urdu, further validation across more diverse linguistic and cultural contexts is planned.
Summary updated