Abstract
ProGuard is a vision-language safety model designed to shift the paradigm of AI moderation from reactive filtering to proactive risk analysis. By identifying and describing novel, out-of-distribution (OOD) risks, ProGuard addresses the limitations of static safety taxonomies. Trained purely via reinforcement learning on a balanced multimodal dataset of 87K samples, the 7B-parameter model significantly outperforms existing open-source safeguards in detecting emerging threats while maintaining high reasoning efficiency.
Introduction: The Need for Proactive Moderation
As generative AI capabilities evolve, the landscape of potential risks becomes increasingly complex and dynamic. Traditional safety "guard" models—external components that filter model inputs and outputs—are typically reactive. They rely on fixed, static taxonomies (e.g., "Hate Speech," "Violence," or "Sexual Content"). When a novel risk emerges that falls outside these pre-defined categories, reactive models often fail to categorize it correctly or miss the threat entirely.
Furthermore, existing multimodal guards often suffer from modality bias, performing well on text but struggling with visual inputs due to skewed training data. ProGuard addresses these challenges by transforming the guard from a simple classifier into an analyst capable of inferring and describing unseen harms.
Methodology: Pure RL and OOD Inference
ProGuard (based on the Qwen2.5-VL-7B architecture) utilizes several novel training strategies to achieve proactive safety:
- Pure Reinforcement Learning (RL): Inspired by the "R1-Zero" approach, ProGuard bypasses standard Supervised Fine-Tuning (SFT). It is trained purely via Group Relative Policy Optimization (GRPO), allowing the model to "self-evolve" its reasoning traces. This results in more concise and efficient "thinking" processes compared to SFT-initialized models.
- OOD Safety Category Inference: To move beyond static lists, the model is trained on tasks where specific categories are intentionally removed from its known taxonomy. The model must proactively infer a plausible name and description for the unseen risk.
- Synonym-Bank-Based Similarity Reward: Rather than penalizing the model for failing to match a specific string exactly, the authors use a sentence transformer and a bank of synonyms. This provides a "smoother" reward signal that encourages semantic understanding over rote memorization.
Balanced Multimodal Datasets
To mitigate modality bias, the researchers constructed a balanced dataset of 87,000 samples. The composition is roughly split between Text-only (33%), Image-only (32%), and Text-Image pairs (34%). This data aggregates ten major safety benchmarks, including GuardReasoner, BeaverTails-V, and MedSafetyBench. Ground truth labels were established through a majority-voting mechanism using high-performing models like InternVL2 and Qwen2.5-VL-72B.
Key Insights and Results
- Proactive Superiority: ProGuard improved OOD risk detection by 52.6% and OOD risk description by 64.8% compared to existing baselines. This demonstrates that training for inference rather than just classification makes the system robust against "jailbreaks" and emerging harms.
- State-of-the-Art Performance: ProGuard-7B sets a new benchmark for open-source safety models, performing comparably to closed-source models like GPT-4o-mini and Gemini 1.5-Flash in prompt classification.
- Reasoning Efficiency: One of the most significant findings was that pure RL training led to more concise reasoning. ProGuard achieved superior results using only ~52 "thinking" tokens, whereas models initialized with SFT required ~114 tokens.
- The Criticality of Modality Balance: The study confirms that a 1:1:1 ratio across text, image, and multimodal data is essential. Without balanced visual data during training, a model's ability to detect visual risks collapses, regardless of its text-based performance.
Conclusion and Future Directions
ProGuard represents a significant step toward autonomous safety systems. While OOD inference remains a challenging task, ProGuard's ability to describe novel risks provides a powerful tool for automated safety taxonomy expansion and risk discovery. Future work will focus on scaling these capabilities to even more complex multi-class classification scenarios and refining the model's ability to differentiate between subtle, overlapping risk types.
Summary updated