Evaluation of Audio Language Models for Fairness, Safety, and Security

Ranya Aloufi, Srishti Gupta, Soumya Shaw, Battista Biggio, Lea Schönherr

This paper, recently published by researchers from CISPA and the University of Cagliari, provides a much-needed structural look at how we evaluate the "trustworthiness" of Audio Large Language Models (ALLMs).

TL;DR

The researchers introduce a new structural taxonomy and evaluation framework for Audio Large Language Models (ALLMs) to address the "black box" problem in current safety testing. Their findings reveal that a model’s architectural choices—how it represents audio and where it performs reasoning—directly dictate its susceptibility to bias, safety failures, and adversarial attacks.

What Problem Does This Paper Address?

As we move from text-only LLMs to models that "hear" (like GPT-4o or Qwen2-Audio), we encounter a major evaluation gap. Current benchmarks often treat ALLMs as a monolith, ignoring the fact that some models transcribe audio to text first (cascaded), while others process raw audio embeddings directly (audio-native).

This lack of structural awareness leads to "conflated analyses." For example, if a model fails to refuse a harmful request delivered via audio, is it because the speech recognition failed, or because the safety guardrails are weaker for the audio modality? The authors argue that without a structure-aware framework, we cannot understand why these models fail, only that they do.

Methods

The authors propose a two-axis taxonomy to categorize ALLMs:

  1. System-Level: Categorized as Cascaded (ASR + LLM), Audio-Text Multimodal (shared model for both), or Audio-Native (reasoning directly on audio).
  2. Representational: Categorized by whether they use Discrete Tokens (quantized units) or Continuous Embeddings (vectors).

To test these models, they developed a unified evaluation framework focusing on Fairness, Safety, and Security (FSS). A key methodological innovation here is the use of controlled audio synthesis. Instead of using "found" audio, they used StyleTTS to generate speech from text prompts, allowing them to precisely vary paralinguistic features (like accent, emotion, and intensity) while keeping the semantic content identical. This isolates whether a model's behavior changes based on how something is said, rather than what is said.

Datasets

The study utilizes a mix of existing semantic benchmarks and acoustic reference datasets to build a massive, controlled testing suite:

  • Semantic Content: LibriSQA for benign prompts and CoSafe for unsafe prompts (covering 14 categories like child abuse, terrorism, and self-harm).
  • Acoustic References: MEAD, VCTK, and FaIST-EV were used to provide the "voice" characteristics for synthesis.
  • The Resulting Benchmark: A dataset comprising 33.6k prompts for fairness (varying 8 accents and 10 emotions), 1,400 for safety, and 2,100 for security (including adversarial perturbations).

Key Insights and Ideas

  • The "Stress Test" of Unsafe Prompts: Models that appear fair and neutral when answering benign questions often "unmask" biased behavior when pushed with unsafe prompts. For instance, a model might use a professional tone for one accent but a dismissive, moralizing tone for another when refusing the same harmful request.
  • Cross-Modal Inconsistency: There is a significant "safety gap" between text and audio. A model might correctly refuse a text-based prompt but comply when that same prompt is spoken, or vice versa. The researchers found a Cross-Modal Safety Inconsistency (CMSI) of about 31% for Qwen2 and 22% for Phi-4.
  • Adversarial "Logistical" Attacks: Unlike text attacks that often aim to flip a label, adversarial audio attacks (like PGD and FGSM) often act as a "denial of service" for the model's reasoning. They significantly increase latency (the time it takes to respond) and cause the model to produce much shorter, truncated responses, even if the model still technically understands the prompt.

Key Results

  • Safety Performance: Phi-4-Multimodal generally showed higher refusal rates (better safety) than Qwen2-Audio, but Qwen2 exhibited higher toxicity variance, meaning its "bad" answers were more inconsistently harmful.
  • Fairness Disparities: The researchers observed heightened safety escalation (refusing more aggressively) for African accents compared to English or American accents under identical prompts in the Phi-4 model.
  • Security Robustness: Under PGD attacks, model latency increased by an average of 7.88 seconds, and response lengths dropped by nearly 80%. However, semantic understanding remained relatively stable (similarity scores > 0.8), suggesting the models are semantically robust but operationally fragile.
  • Judge Reliability: The study used an "LLM-as-a-judge" (GPT-OSS-Safeguard-120b), which achieved an 85.33% agreement rate with human expert annotators.

Limitations and Future Work

The authors acknowledge that their empirical analysis was limited to two specific models (Qwen2 and Phi-4), both of which fall into the "Audio-Text Multimodal" category. They suggest that future work should expand to:

  • Fully Cascaded and Purely Audio-Native architectures to see if the trends hold.
  • Multi-turn interactions, as safety risks often compound over a conversation.
  • More diverse acoustic environments, such as background noise or multi-speaker scenarios, which were not covered in this controlled synthesis study.

Summary updated

Abstract

Audio large language models (ALLMs) have recently advanced spoken interaction by integrating speech processing with large language models. However, existing evaluations of fairness, safety, and security (FSS) remain fragmented, largely because ALLMs differ fundamentally in how acoustic information is represented and where semantic reasoning occurs. Differences that are rarely made explicit. As a result, evaluations often conflate structurally distinct systems, obscuring the relationship between model design and observed FSS behavior. In this work, we introduce a structural taxonomy (system-level and representational) of ALLMs that categorizes systems along two axes: the form of audio input representation (e.g., discrete vs. continuous) and the locus of semantic reasoning (e.g., cascaded, multimodal, or audio-native). Building on the taxonomy, we propose a unified evaluation framework that assesses semantic invariance under paralinguistic variation, refusal and toxicity behavior under unsafe prompts, and robustness to adversarial audio perturbations. We apply this framework to two representative systems and observe systematic differences in refusal rates, attack success, and toxicity between audio and text inputs. Our findings demonstrate that FSS behavior is tightly coupled to how acoustic information is integrated into semantic reasoning, underscoring the need for structure-aware evaluation of audio language models.
BibTeX
@misc{aloufi2026evaluation,
  author = {Ranya Aloufi and Srishti Gupta and Soumya Shaw and Battista Biggio and Lea Schönherr},
  title = {{Evaluation of Audio Language Models for Fairness, Safety, and Security}},
  year = {2026},
  eprint = {2603.13262},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2603.13262},
}
Figure