Introduction
Current Large Language Model (LLM) safety alignment typically follows a "one-size-fits-all" approach. Whether a query comes from a medical professional or a layperson, the model applies the same rigid safety filters. This creates a persistent tension: overly restrictive models hinder experts, while overly permissive models pose risks to the public. Furthermore, existing safety mechanisms are brittle and often bypassed by "jailbreak" prompts that trick the model into ignoring its training.
sudoLLM addresses this by introducing a "user-aware" alignment framework. It allows models to provide sensitive information to authorized experts (Alice) while refusing the same queries from unauthorized users (Bob).
Methodology
The sudoLLM paradigm operates in two distinct stages: training and inference.
1. Biased Query Rephrasing
When a user submits a query, it is first processed by a trusted Small Language Model (SLM). This SLM re-writes the query to maintain its original meaning while injecting a subtle "bias signal." It achieves this by slightly favoring specific sets of rare tokens: one set ($V_{Alice}$) for authorized experts and another ($V_{Bob}$) for unauthorized users. This creates a detectable distribution shift that is nearly invisible to humans but clear to the model.
2. Biased Fine-Tuning (BFT)
The target LLM is trained on these rephrased queries to learn a simple rule: if the query contains the "Alice" signal, provide a helpful, potentially sensitive answer. If it contains the "Bob" signal, provide a standard refusal. This effectively transforms the complex task of detecting harmful intent into a simpler task of distribution detection.
Key Insights and Results
- Resolving Next-Token Tension: sudoLLM resolves the conflict between a model's objective to predict the next likely word and the requirement to maintain safety by providing a clear signal in the input that dictates the operational mode.
- Superior Alignment: Biased Fine-Tuning (BFT) improved refusal accuracy for unauthorized users by an average of 49.2% over standard fine-tuning.
- Jailbreak Immunity: Because safety is baked into the input distribution rather than just output tokens, sudoLLM showed a 13x improvement in robustness against state-of-the-art prefix-based jailbreaks compared to GPT-4o.
- Generalization and Quality: The model generalizes well to unseen datasets, showing a ~73% improvement over baselines in out-of-distribution safety performance. Semantic similarity tests confirm that the rephrasing process maintains high fidelity (~0.9 cosine similarity) to the original user intent.
Limitations and Future Work
The security of sudoLLM relies on the integrity of the "Secure Environment" where the SLM and authentication occur. If a user can bypass the SLM to send raw queries directly to the LLM, the system could be compromised. Future research will focus on scaling this approach to models with 100B+ parameters and expanding the training datasets to further refine role distinction without impacting performance on non-restricted content.
Summary updated