TombRaider: Entering the Vault of History to Jailbreak Large Language Models

Jianhua Ding, Jiahao Zhang, Yi Liu, Zhangwei Ding, Gelei Deng, Yuekang Li

TL;DR

TombRaider is a novel multi-turn jailbreak attack that exploits an LLM’s vast internal store of historical, artistic, and cultural knowledge to bypass safety filters. By framing harmful requests as benign inquiries into historical facts or literary analysis, it achieves near 100% success rates on leading models like GPT-4o and Claude-3.5, even when active defense mechanisms are in place.

What Problem Does This Paper Address?

Most existing "jailbreak" attacks rely on surface-level prompt manipulation—like obfuscating intent or using complex syntactic perturbations—to trick a model into ignoring its safety training. However, modern safety filters are increasingly adept at catching these patterns.

The authors of TombRaider identify a deeper, more fundamental vulnerability: Large Language Models (LLMs) are trained on massive datasets that inevitably include detailed accounts of illegal, unethical, or dangerous historical events (e.g., famous bank robberies or chemical warfare). While a model might refuse a direct request for "how to rob a bank," it is much less likely to refuse a request for "historical details on the tactics of Herman Karl Lamm." The paper addresses the challenge of systematically eliciting this "latent harmful knowledge" by exploiting the model's tendency to be less defensive when engaging with seemingly factual or academic contexts.

Methods

TombRaider employs a sophisticated multi-turn, agent-based architecture consisting of two cooperative agents:

  1. The Inspector Agent: This agent takes a user-provided keyword (e.g., "bank robbing") and constructs a benign, historically grounded prompt. It ensures the conversation remains semantically coherent and anchors the dialogue in "safe" territory (history or art).
  2. The Attacker Agent: This agent takes the output from the Inspector and the target model's response to generate increasingly adversarial queries. It "drills deeper" into the specific methods described in the historical context, gradually steering the model toward contemporary, actionable harmful content.

The attack follows a Prompt Progression Strategy:

  • Round 1 (Historical Context): Establishes a benign factual base.
  • Round 2 (Focused Reframing): Identifies the most "provocative" or "semantically rich" entities from the model's response.
  • Round 3+ (Semantic Escalation): Uses speculative or morally ambiguous cues (e.g., "Could this apply today?") to bridge the gap between historical fact and modern-day harm.

The authors define three specific attack modes: Figures Jailbreak (exploiting historical figures), Artistic Jailbreak (exploiting literary/artistic works), and Debate Jailbreak (using formal philosophical defense of harmful positions).

Datasets

The study introduces a new dataset categorized into 12 harmful scenarios, including Illegal Activity, Malware Generation, and Political Lobbying. This provides a more granular taxonomy than previous benchmarks.

For evaluation, the authors used:

  • AdvBench: A standard benchmark of curated adversarial prompts.
  • Custom Historical Dataset: Mapped to the 12 scenarios to test the specific TombRaider methodology.

Key Insights and Ideas

  • History as a Trojan Horse: The "aha moment" is that LLMs view historical and artistic content as "safe" facts. By tethering a harmful request to a historical anchor, the attacker can bypass the "intent hiding" phase that most filters look for.
  • Statelessness is a Liability: Current safety filters are often optimized for single-turn, isolated queries. TombRaider exploits the lack of long-horizon context tracking; by the time the model realizes it's providing harmful info, it has already committed to the "historical analysis" persona.
  • Knowledge vs. Refusal: The paper argues that jailbreaks succeed not just because of prompt tricks, but because the harmful knowledge is already there in the model's parameters. The attack is simply a retrieval mechanism.

Key Results

The researchers evaluated TombRaider on six major models: GPT-4o, Claude-3.5, DeepSeek-v3, Llama3.2, Qwen2.5, and Gemma2.

  • Near-Perfect Success: On "bare" models (without additional external defenses), TombRaider achieved nearly 100% Attack Success Rate (ASR) across almost all scenarios.
  • Resilience to Defenses: Against active defenses like "self-reminders" and "in-context demonstrations," TombRaider maintained an ASR of over 55.4%, significantly outperforming state-of-the-art baselines like PAIR and RedQueen, which often dropped below 10%.
  • Efficiency: Most successful jailbreaks occurred within five rounds of dialogue, making it a highly practical and efficient attack vector.

Limitations and Future Work

The authors acknowledge several limitations:

  • Model Evolution: While effective on current SOTA models, its performance on future architectures with more integrated, history-aware safety layers is unknown.
  • Language Constraints: The evaluation was primarily limited to English and the authors' native languages.
  • Controlled Environment: The study relies on controlled red-teaming experiments rather than real-world "in-the-wild" validation.

Future Work suggests moving beyond surface-level filters toward training-time mitigation (e.g., specifically annotating and "unlearning" harmful knowledge during pretraining) and developing dynamic, context-aware refusal strategies that can track adversarial intent across long conversations.

Summary updated

Abstract

Warning: This paper contains content that may involve potentially harmful behaviours, discussed strictly for research purposes.Jailbreak attacks can hinder the safety of Large Language Model (LLM) applications, especially chatbots.Studying jailbreak techniques is an important AI red teaming task for improving the safety of these applications.In this paper, we introduce TOMBRAIDER, a novel jailbreak technique that exploits the ability to store, retrieve, and use historical knowledge of LLMs.TOMBRAIDER employs two agents, the inspector agent to extract relevant historical information and the attacker agent to generate adversarial prompts, enabling effective bypassing of safety filters.We intensively evaluated TOMBRAIDER on six popular models.Experimental results showed that TOMBRAIDER could outperform state-of-the-art jailbreak techniques, achieving nearly 100% attack success rates (ASRs) on bare models and maintaining over 55.4% ASR against defence mechanisms.Our findings highlight critical vulnerabilities in existing LLM safeguards, underscoring the need for more robust safety defences.
BibTeX
@misc{ding2025tombraider,
  author = {Jianhua Ding and Jiahao Zhang and Yi Liu and Zhangwei Ding and Gelei Deng and Yuekang Li},
  title = {{TombRaider: Entering the Vault of History to Jailbreak Large Language Models}},
  year = {2025},
  eprint = {2501.18628},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2501.18628},
}
Figure