A Comparative Analysis of Ethical and Safety Gaps in LLMs using Relative Danger Coefficient

Yehor Tereshchenko, Mika HämäläinenNAACL 2025

TL;DR

This paper introduces the Relative Danger Coefficient (RDC), a novel metric for quantifying the ethical and safety risks posed by Large Language Models (LLMs). Through both manual and automated testing, the authors reveal significant inconsistencies in LLM safety across various models and prompt types, highlighting the critical need for robust human oversight and continuous refinement of AI moderation systems.

What Problem Does This Paper Address?

As LLMs become increasingly integrated into critical decision-making domains like healthcare and criminal justice, ensuring their ethical alignment with human values is paramount. However, current methods for evaluating LLM safety often fall short, failing to capture the nuanced ways these models can generate harmful, biased, or discriminatory content, especially when faced with adversarial or cleverly rephrased prompts. This paper addresses this gap by proposing a more comprehensive metric and conducting a comparative analysis of leading LLMs to identify their ethical and safety vulnerabilities.

Methods

The authors employed a two-pronged approach:

  • Manual Testing: A variety of prompts, categorized by potential harm (e.g., hazardous information, hate speech, ethical dilemmas), were used to stress-test LLMs. These prompts were designed to be simple, complex, and even adversarial, sometimes employing role-playing or educational framing to probe for vulnerabilities. The responses were then qualitatively analyzed.
  • Algorithmic Testing: A novel metric, the Relative Danger Coefficient (RDC), was developed. The RDC quantifies LLM safety by categorizing responses into four levels: Good (G), Uncertain (U), Partially Unsafe (P), and Directly Unsafe (D). It then applies weighting factors to these categories and incorporates penalty terms for consistency, severity, repetition vulnerability, and adversarial exploitability. This results in a single score from 0-100, where a higher score indicates greater danger.

Datasets

The paper does not explicitly mention training datasets. For evaluation, the authors created a comprehensive set of prompts across various categories, including:

  • Hazardous Information: Explosive device instructions, historical warfare tactics.
  • Hate Speech and Discrimination: Religious fear-mongering, statistical racial comparisons, logical puzzle embedding discrimination.
  • Ethical Dilemmas: Trolley problems with variations (basic, special groups, scientist sacrifice), intellectual property vs. human life, high-stakes AGI dilemmas.

These prompts were designed to be reformatted and tested in multiple ways to assess model robustness.

Key Insights and Ideas

  • The "Reasoning" Paradox: While advanced reasoning capabilities in LLMs can lead to more nuanced and helpful responses, they can also inadvertently create new avenues for generating unsafe content or justifying ethically questionable outputs.
  • Adversarial Prompts are Key: Simple refusals are insufficient. Clever rephrasing, role-playing, or framing prompts as educational or historical can often bypass safety filters, revealing significant vulnerabilities.
  • Inconsistency is a Major Risk: Models that sometimes refuse a harmful request but then provide partial or direct instructions under slightly different conditions pose a significant threat. This inconsistency makes them unpredictable and exploitable.
  • No LLM is Immune: The study strongly suggests that no single LLM is completely immune to adversarial exploitation, especially when prompts are subtle or repeated.
  • Human Oversight is Non-Negotiable: The findings underscore the critical need for continuous human-in-the-loop oversight and iterative refinement of AI moderation systems, particularly in high-stakes applications.

Key Results

  • Manual Testing: Revealed inconsistencies in LLM refusal patterns. Models sometimes provided precise instructions for dangerous devices (e.g., GPT-4 in one scenario), while others offered vague hints or partial information. Ethical dilemmas often exposed biases, with some models prioritizing abstract benefits over human lives or struggling with nuanced scenarios involving vulnerable groups.
  • Algorithmic Testing (RDC): The RDC metric allowed for systematic comparison. High RDC scores were observed for categories like Substance-Drug and Weapon-related queries across models. Gemini models generally performed better on ethical issues, while DeepSeek showed higher risk profiles in certain discrimination scenarios. Newer models generally showed improved ethics and security on average compared to older versions.

Limitations and Future Work

The authors acknowledge several limitations:

  • Limited Scope of Models: The study focused on a specific set of LLMs (GPT variants, Gemini, DeepSeek).
  • Prompt-Based Evaluation: Reliance on prompt-based testing might not capture all potential failure modes.
  • Potential Subjectivity: Manual analysis can introduce subjectivity.
  • Novel Metric: The RDC is a new metric, and its long-term validity and interpretability will require further validation.
  • Text-Only Focus: The study did not evaluate multimodal capabilities.
  • Resource Constraints: Manual testing was conducted by a single annotator.
  • Dynamic Nature of LLMs: Frequent model updates can impact reproducibility.

Future work should involve incorporating multiple annotators, exploring multimodal LLM safety, and conducting longitudinal studies to track improvements and evolving vulnerabilities as models are continuously updated.

Summary updated

BibTeX
@inproceedings{tereshchenko2025comparative,
  author = {Yehor Tereshchenko and Mika Hämäläinen},
  title = {{A Comparative Analysis of Ethical and Safety Gaps in LLMs using Relative Danger Coefficient}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2025},
}
Figure