METAL: Metamorphic Testing Framework for Analyzing Large-Language Model Qualities

Sangwon Hyun, Mingyu Guo, Muhammad Ali Babar

TL;DR

The METAL framework introduces a systematic way to evaluate Large Language Model (LLM) qualities—like robustness, fairness, and efficiency—using Metamorphic Testing. By shifting the focus from simple accuracy to the relationship between outputs of perturbed inputs, METAL reveals hidden quality risks in popular models like ChatGPT, Llama2, and GooglePaLM without requiring massive labeled datasets.

What Problem Does This Paper Address?

As LLMs become the backbone of modern software, their "black-box" nature presents a major challenge: how do we know they are actually reliable? Traditional testing relies on "test oracles"—knowing the exact right answer for every input. However, for complex generative tasks, a single "correct" answer often doesn't exist, and creating millions of labeled examples is prohibitively expensive.

Existing evaluation methods often suffer from:

  1. Limited Scope: They focus mostly on accuracy or a single type of robustness.
  2. The Oracle Problem: They struggle to evaluate generative outputs (like summaries) where the "correct" answer is subjective.
  3. Simplistic Metrics: Most use Attack Success Rate (ASR), which doesn't account for whether the "adversarial" input is even readable or semantically valid.

Methods

The authors propose METAL (MEtamorphic Testing for Analyzing LLMs). The core idea is Metamorphic Testing (MT): instead of checking if an output is "correct," you check if the relationship between outputs of related inputs is "consistent."

Key Components:

  • Metamorphic Relations (MRs): These are the "rules" of the test. For example, if you swap a few characters in a sentence (robustness) or change the user's perceived gender (fairness), the model's sentiment analysis should remain the same.
  • Automated Perturbation: The framework uses 13 types of perturbations across character, word, and sentence levels (e.g., swapping characters, replacing synonyms, or adding demographic context).
  • New Metric (EFM): The authors argue that ASR is a poor metric if the perturbation turns the text into gibberish. They introduce Effectiveness of MRs (EFM), which multiplies ASR by a "PerturbQuality" score (based on semantic similarity) to ensure the tests are actually meaningful.
  • Self/Cross-Examination: They explored using LLMs themselves to generate these perturbations, finding that ChatGPT is particularly effective at "stress-testing" other models.

Datasets

The study didn't train new models but evaluated three industry leaders: GooglePaLM (text-bison-001), ChatGPT (GPT-3.5-Turbo), and Llama2 (7b-chat).

They sampled 900 input texts from diverse sources:

  • Amazon reviews (for sentiment analysis).
  • News articles and headings (for classification and summarization).
  • Kaggle toxicity datasets.
  • General web sources for Q&A and Information Retrieval (IR).

Key Insights and Ideas

  • The "Oracle" isn't necessary: By defining relations (e.g., "Output A should equal Output B"), METAL can generate hundreds of test cases automatically from a single unlabeled input.
  • Not all "Attacks" are equal: A model "failing" because the input was turned into unreadable noise isn't a bug; it's expected. The EFM metric is a crucial step toward more realistic AI safety testing.
  • Llama2’s "Confusion" in IR: A fascinating insight was that Llama2 showed high non-determinism in Information Retrieval. When it didn't understand a prompt, it would return different "canned" responses or recommendations each time, leading to high variance and lower robustness.
  • Task-Specific Vulnerabilities: Character-level swaps (like typos) are devastating for classification tasks, while word-level changes (like synonyms) are more effective at breaking generative tasks like summarization.

Key Results

The researchers evaluated the models across six tasks (Toxicity, Sentiment, News, Q&A, Summarization, IR) and four Quality Attributes (Robustness, Fairness, Non-determinism, Efficiency).

  • Top Performer: GooglePaLM generally outperformed the others, showing the highest robustness and fairness across most tasks.
  • Robustness Gap: Classification tasks (like sentiment analysis) were generally more robust than generative tasks (like Q&A). Llama2 struggled significantly with robustness in Information Retrieval, hitting an ASR near 1.0 (meaning it failed almost every perturbation test).
  • Fairness: All models showed some level of bias when demographic markers were added, but GooglePaLM was the most consistent.
  • Efficiency: GooglePaLM was the most stable in terms of inference time, whereas ChatGPT showed the highest variance, especially on generative tasks.

Limitations and Future Work

While METAL is a significant step forward, the authors note a few areas for growth:

  • Attribute Coverage: The current version doesn't yet cover Security (e.g., prompt injection), Privacy, or Explainability.
  • Model Scale: The study used the 7B version of Llama2 due to hardware constraints; larger versions might yield different results.
  • Prompt Engineering: The framework relies on specific prompts to generate perturbations. Future work could involve more sophisticated "prompt-based" attacks to see how models handle subtle instructional changes.

By providing an open-source framework, the authors hope to help engineers "stress-test" their LLM-based applications before they hit production.

Summary updated

Abstract

Large-Language Models (LLMs) have shifted the paradigm of natural language data processing. However, their black-boxed and probabilistic characteristics can lead to potential risks in the quality of outputs in diverse LLM applications. Recent studies have tested Quality Attributes (QAs), such as robustness or fairness, of LLMs by generating adversarial input texts. However, existing studies have limited their coverage of QAs and tasks in LLMs and are difficult to extend. Additionally, these studies have only used one evaluation metric, Attack Success Rate (ASR), to assess the effectiveness of their approaches. We propose a MEtamorphic Testing for Analyzing LLMs (METAL) framework to address these issues by applying Metamorphic Testing (MT) techniques. This approach facilitates the systematic testing of LLM qualities by defining Metamorphic Relations (MRs), which serve as modularized evaluation metrics. The METAL framework can automatically generate hundreds of MRs from templates that cover various QAs and tasks. In addition, we introduced novel metrics to assess the effectiveness of MRs accurately by integrating the ASR method into the semantic qualities of text. Through the experiments conducted with three prominent LLMs, we have confirmed that the METAL framework effectively evaluates essential QAs on primary LLM tasks and reveals the quality risks in LLMs. Moreover, the newly proposed metrics can guide the optimal MRs for testing each task and suggest the most effective method for generating MRs.
BibTeX
@inproceedings{hyun2024metal,
  author = {Sangwon Hyun and Mingyu Guo and Muhammad Ali Babar},
  title = {{METAL: Metamorphic Testing Framework for Analyzing Large-Language Model Qualities}},
  booktitle = {ICST},
  year = {2024},
  url = {https://openreview.net/forum?id=nuVovoiSHo},
}

Loading related content...

Figure