TL;DR
This paper introduces ClarityEthic, a novel framework that enhances LLM moral judgment by having them emulate human decision-making through social norms. It leverages LLMs' reasoning to uncover conflicting norms and uses contrastive learning to select the most reliable one, leading to more accurate and explainable moral judgments.
What Problem Does This Paper Address?
Large Language Models (LLMs) are increasingly integrated into our lives, making their safety and ethical behavior paramount. However, current methods for assessing LLM moral valence (whether they support or oppose an action) are often untrustworthy and lack explainability. A key challenge is that human moral decisions are complex, often involving conflicting social norms. For instance, conserving resources might conflict with maintaining hygiene. LLMs need to identify the most appropriate norm in a given context to make sound moral judgments. This paper addresses the need for LLMs to not only make moral judgments but also to provide clear, human-understandable explanations for them, mirroring how humans navigate complex ethical situations.
Methods
ClarityEthic employs a two-stage training framework:
- Pre-training Task-Specific Language Models: The authors first pre-train three distinct T5-based models: a classifier for binary moral judgment, a rationale generator, and a norm generator. This stage leverages LLMs' reasoning capabilities by prompting them to generate rationales that link human actions to social norms. This is done using a two-step prompting strategy to elicit moral tension and judgment rationales.
- Fine-tuning with Contrastive Learning: The rationale and norm generators are then fine-tuned using contrastive learning. This involves organizing training data into triplets of moral and immoral actions associated with the same social norm. The goal is to train the model to generate similar norms for actions under the same norm while distinguishing them from actions under different norms, thereby enhancing discriminative feature learning.
During inference, ClarityEthic generates both moral and immoral paths of rationales and norms, then uses a filtering mechanism to select the most reliable one to support the final moral judgment.
Datasets
The authors evaluate ClarityEthic on two publicly available datasets:
- Moral Stories (Emelin et al., 2020): This dataset contains structured descriptive morality scenarios with pairwise moral and immoral actions within the same social norm.
- ETHICS (Hendrycks et al., 2020): This dataset comprises scenarios annotated with five moral concepts: justice, deontology, virtue ethics, utilitarianism, and commonsense intuitions. For their work, they focus on justice, virtue, and deontology, using the concept definitions as ground-truth norms.
Key Insights and Ideas
- Emulating Human Moral Strategy: The core idea is that by mimicking how humans navigate conflicting social norms, LLMs can achieve more robust and explainable moral judgments.
- Contrastive Learning for Norm Selection: Using contrastive learning to differentiate between actions within the same norm and those under different norms is crucial for selecting the most relevant and reliable social norm.
- LLM Rationales as Intermediate Steps: Leveraging LLMs to generate rationales for social norms provides a powerful way to imbue the models with commonsense knowledge and improve norm generation.
- Explainability Through Norms: The generated social norms serve as direct explanations for the LLM's moral judgments, increasing transparency and trustworthiness.
Key Results
- Classification Performance: ClarityEthic achieved state-of-the-art performance on both the Moral Stories and ETHICS datasets, outperforming strong baselines including GPT-4 and specialized models like MoralCoT. For instance, on the Moral Stories dataset, ClarityEthic achieved an accuracy and macro-F1 score of 0.845, outperforming other models.
- Norm Generation Quality: Human evaluations showed that ClarityEthic's generated norms were highly plausible and relevant, comparable to or better than those generated by strong LLMs like GPT-3.5. Automatic metrics like BLEU and ROUGE-L also indicated superior performance.
- Significance of Components: Ablation studies demonstrated the importance of both the fine-tuning module and contrastive learning for achieving high performance. The inclusion of contrastive learning significantly improved classification results.
Limitations and Future Work
- Cultural Bias: The training data is primarily derived from US norms, which may limit applicability to other cultural contexts. The user study also focused on Asian participants, potentially introducing bias. Future work aims to address this by creating culturally tailored benchmarks.
- Moral Intensity and Cross-Culture: The current version does not explicitly focus on moral intensity or cross-cultural moral variations, though the approach might implicitly capture moral intensity. Future research will explore these aspects more deeply.
- Multi-Party Scenarios: ClarityEthic currently handles two opposite paths (moral/immoral). Extending it to multi-party scenarios would require new datasets and strategies.
- Internal Model Interpretability: While the approach provides explanations for its judgments, it doesn't delve into the model's internal mechanisms. Future work will focus on enhancing this internal interpretability.
- Broader Impact: The authors emphasize that ClarityEthic should not be used to guide real-world human interactions or provide advice. They also note concerns about potential ethical issues with LLM-generated rationales, despite efforts to curate unbiased data.
Summary updated