Introduction
As Large Language Models (LLMs) evolve, the paradigm of "thinking before speaking"—known as Chain of Thought (CoT) reasoning—is being extended to Audio Reasoning Models (ARMs). These models process complex audio inputs and reason through them to generate responses. However, while reasoning improves general task performance, its implications for safety in the audio domain remain largely unexplored. This paper identifies a critical vulnerability in ARMs: audio jailbreak attacks that leverage "representation drift" to bypass safety guardrails. To address this, we introduce Rebellion, a noise-robust reasoning training method designed to maintain safety even under adversarial conditions.
The Challenge of Representation Drift
Standard Reasoning Training (RT) involves fine-tuning models on a mixture of benign and safety-related reasoning data. While this protects models against simple harmful queries, it fails against advanced audio jailbreaks like Advwave. These attacks inject optimized adversarial noise into the audio signal, causing representation drift. This drift shifts the model's internal activations away from the "harmful" cluster and toward a "complying" cluster, effectively blinding the safety guardrails and forcing the model to generate harmful content.
Rebellion: Robust Reasoning Training
To counter representation drift, Rebellion employs a min-max optimization strategy, conceptually similar to adversarial training. The training process involves two key steps:
- Inner Problem (Maximizing Loss): The model simulates the worst-case representation drift—a noise vector that maximizes safety loss—using a one-step gradient approximation.
- Outer Problem (Minimizing Loss): The model's weights are updated to minimize the combined safety and benign loss while under the influence of this simulated worst-case drift.
By training the model to recover from the most disruptive internal perturbations, Rebellion ensures that safety reasoning remains robust regardless of adversarial audio noise.
The "Think Twice" Phenomenon
One of the most compelling findings is the emergent "think twice" behavior in Rebellion-trained models. Attacks like Advwave often force a model to begin its response with an affirmative prefix like "Sure." In standard models, this prefix acts as a trap, compelling the model to complete the harmful request.
As shown in Figure 1, a Rebellion-trained model may still output the forced "Sure," but its robust safety reasoning then kicks in. The model generates a CoT that recognizes the policy violation and ultimately issues a refusal. It effectively "catches" its own forced error before delivering the final answer.
Experimental Results
Experiments conducted on the Qwen2-Audio-7B model demonstrate that Rebellion significantly enhances safety without compromising utility. Against the Advwave jailbreak, standard RT yielded a Harmful Score (HS) of 26.25%, whereas Rebellion reduced this to just 1.25%. Furthermore, Rebellion completely neutralized rephrasing attacks, dropping the HS from over 50% to 0%.
Crucially, these safety gains do not come at the cost of general intelligence. As seen in Table 1, benign accuracy on benchmarks like GSM8K and MMLU-biology remains stable or even improves under moderate Rebellion training intensities. However, there is a sensitivity threshold; if the simulated noise intensity is set too high, the optimization becomes overly difficult, leading to a decline in benign performance.
Conclusion
Rebellion represents a significant step forward in securing Audio Reasoning Models. By addressing the root cause of audio jailbreaks—representation drift—it enables models to maintain their safety integrity even when forced into initial compliance. Future work will explore extending this robust training framework from Supervised Fine-Tuning to Reinforcement Learning-based reasoning paradigms.
Summary updated