All You Need Is S P A C E: When Jailbreaking Meets Bias Audit and Reveals What Lies Beneath the Guardrails (Student Abstract)

Arka Dutta, Aman Priyanshu, Ashiqur R. KhudaBukhshAAAI 2025 · DOI

TL;DR

This paper introduces a novel method for auditing Large Language Models (LLMs) for biases by combining a recent jailbreaking technique with a simplified bias audit framework. Their findings reveal that Llama 3, when jailbroken, exhibits significant antisemitism, racism, misogyny, and homophobia, similar to other LLMs previously found susceptible to such biases.

What Problem Does This Paper Address?

As LLMs become more sophisticated and integrated into daily applications, ensuring their safety and fairness, especially for disadvantaged minorities, is paramount. A key challenge in auditing LLMs for biases is the "cat-and-mouse" game: as soon as a vulnerability is discovered and patched, newer models may no longer exhibit that specific bias. This makes existing audit frameworks quickly outdated. While previous work, like the "Toxicity Rabbit Hole" (TRH) framework, has uncovered biases in many LLMs, the Llama family of models remained relatively underexplored in this regard. This paper aims to address this gap by developing a method to effectively audit Llama 3 for biases, even as its safety guardrails evolve.

Methods

The authors propose a novel approach that merges two existing lines of research: bias auditing and LLM jailbreaking. They adapt the "Toxicity Rabbit Hole" (TRH) framework, which iteratively prompts an LLM to generate increasingly toxic content based on initial stereotypes. To enhance the effectiveness of this audit, they integrate a recently discovered jailbreaking technique for Llama 3. This technique involves presenting prompt characters as space-separated characters, which makes Llama 3 more susceptible to generating harmful responses. The researchers then compare the outputs of a jailbroken Llama 3 with a non-jailbroken version (acting as a control) to identify and quantify biases.

Datasets

The paper does not explicitly mention training datasets. For evaluation, they created a dataset of 20,000 Llama 3 responses elicited using their novel jailbreaking and bias auditing approach, targeting 20 specific identity groups. Additionally, they provide another 20,000 responses from non-jailbroken Llama 3 and Llama 2 as a control. The identity groups were selected by combining the most targeted groups from the original TRH study and the Social Bias Frames (SBIC) dataset.

Key Insights and Ideas

  • Jailbreaking as an Audit Accelerator: The core insight is that by combining a known jailbreaking technique with a bias audit framework, researchers can more effectively expose latent biases in LLMs, even those with seemingly robust guardrails.
  • Llama 3's Vulnerability: Despite advancements in LLM safety, Llama 3, like many predecessors, is shown to be susceptible to generating highly offensive and harmful content when its safety mechanisms are bypassed.
  • Outcome Homogenization: The study suggests that the toxic outputs generated by Llama 3 are strikingly similar to those produced by other well-known LLMs, raising concerns about "algorithmic monoculture" and a homogenization of biases across different models.
  • The "Cat-and-Mouse" Game Continues: The findings underscore the ongoing challenge of LLM safety, where new vulnerabilities are constantly being discovered, and existing audit methods need to adapt to keep pace with evolving model architectures and defenses.

Key Results

  • A jailbroken Llama 3 generates significantly more offensive and dangerous content compared to its non-jailbroken counterpart.
  • The study found that Llama 3 is not a substantial improvement over Llama 2 in terms of safety, as evidenced by the biases revealed through their audit.
  • Offensive content related to physical violence against a community rose from 55.81% in base Llama 3 to 66.03% when jailbroken.
  • The percentage of collective violence-related terms in toxic generations increased by 4.04% in jailbroken Llama 3.
  • The audit revealed shocking antisemitism, racism, misogyny, and homophobia in Llama 3's responses, mirroring biases found in other LLMs.

Limitations and Future Work

The authors acknowledge that their work focuses on a specific jailbreaking technique and a simplified version of the TRH framework. Future work could involve exploring other jailbreaking methods and more comprehensive bias audit frameworks. They also suggest that LLMs are like "proverbial volcanoes that might erupt anytime if jailbroken," highlighting the need for continuous research into LLM safety and the underlying data they are trained on.

Summary updated

BibTeX
@inproceedings{dutta2025all,
  author = {Arka Dutta and Aman Priyanshu and Ashiqur R. KhudaBukhsh},
  title = {{All You Need Is S P A C E: When Jailbreaking Meets Bias Audit and Reveals What Lies Beneath the Guardrails (Student Abstract)}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2025},
  doi = {10.1609/aaai.v39i28.35249},
}
Figure