Safety Evaluation and Enhancement of DeepSeek Models in Chinese Contexts

Wenjing Zhang, Xuejiao Lei, Zhaoxiang Liu, Limin Han, Jiaojiao Zhao, Beibei Huang, Zhenhong Long, Junting Guo, Meijuan An, Rongjia Du, Ning Wang, Kai Wang, Shiguo LianarXiv 2025 · DOI · OpenReview

TL;DR

This paper tackles the critical safety shortcomings of the popular DeepSeek-R1 large language model, particularly in Chinese contexts. The authors conduct a thorough safety evaluation of its distilled versions, identify vulnerabilities, and then implement targeted safety enhancements, resulting in models that are significantly safer without sacrificing reasoning abilities.

What Problem Does This Paper Address?

The DeepSeek-R1 model, lauded for its impressive reasoning and open-source nature, has been found to have significant safety issues. Previous research has shown a 100% attack success rate on harmful prompts, and security firms have identified critical vulnerabilities. While some work has addressed these issues in Chinese contexts, the safety of the distilled models within the R1 series, which are crucial for practical deployment due to their efficiency, remained largely unevaluated. This paper aims to fill that gap by comprehensively assessing the safety of these distilled models in Chinese contexts and then developing methods to enhance their safety.

Methods

The researchers employed a two-pronged approach:

  1. Safety Evaluation: They used the comprehensive Chinese safety benchmark, CHiSafetyBench, to evaluate the safety capabilities of six distilled DeepSeek-R1 models. This involved assessing their ability to identify and refuse harmful content across various dimensions like discrimination, violation of values, and infringement of rights.
  2. Safety Enhancement: Based on the identified vulnerabilities, they implemented supervised fine-tuning. This involved creating a custom fine-tuning dataset of approximately 50,000 samples, including instructions for critical safety areas and chain-of-thought reasoning samples to balance safety with performance. This fine-tuning was applied to the entire DeepSeek-R1 model series.

Datasets

  • CHiSafetyBench: This is a comprehensive Chinese safety benchmark used for evaluating the models' safety capabilities. It includes tasks for risk content identification (multiple-choice questions) and risky question refusal.
  • Custom Fine-tuning Dataset: Approximately 50,000 samples were created, comprising instructions for critical safety areas (discrimination, infringement) and chain-of-thought reasoning samples.

Key Insights and Ideas

  • Distillation's Double-Edged Sword: While distillation makes models more efficient, it can inadvertently degrade their safety performance, especially in specific linguistic contexts like Chinese.
  • Chinese Context Matters: Safety vulnerabilities can be amplified or manifest differently in non-English contexts, necessitating specialized evaluation and enhancement.
  • Targeted Enhancement is Effective: Supervised fine-tuning with carefully curated safety data can significantly improve model safety without a substantial hit to reasoning capabilities.
  • Open-Sourcing for Community Benefit: Releasing the safety-enhanced models provides a valuable resource for the research community to build upon.

Key Results

  • Safety Degradation Post-Distillation: The study found that distilled models generally showed a declining trend in safety performance compared to their base models, particularly in risk content identification and refusal of risky questions.
  • Significant Safety Improvements After Enhancement: The safety-enhanced models demonstrated substantial improvements. For instance, in the risk content identification task, accuracy improved by 10.72%. In the refusal-to-answer task, RR-1 and RR-2 increased by 18.39% and 14.72% respectively, while HR decreased by 1.45%.
  • Reasoning Capabilities Maintained: Crucially, the safety enhancements did not significantly impair the models' reasoning abilities. In some cases, reasoning performance even saw marginal improvements.
  • Open-Sourced Models: The authors have open-sourced their safety-enhanced DeepSeek-R1 models to foster further research.

Limitations and Future Work

  • Model Size and Data Influence: The paper notes that improvements in safety performance for some smaller models after distillation might be related to their size and the specific Chinese instructions used in the SFT data.
  • Continued Vigilance: While significant improvements were made, the paper implies that ongoing research and optimization of DeepSeek models are necessary.
  • Future Research: The open-sourced models are intended to serve as a valuable resource for future research and collaborative optimization.

Summary updated

Abstract

DeepSeek-R1, renowned for its exceptional reasoning capabilities and open-source strategy, is significantly influencing the global artificial intelligence landscape. However, it exhibits notable safety shortcomings. Recent research conducted by Robust Intelligence, a subsidiary of Cisco, in collaboration with the University of Pennsylvania, revealed that DeepSeek-R1 achieves a 100\% attack success rate when processing harmful prompts. Furthermore, multiple security firms and research institutions have identified critical security vulnerabilities within the model. Although China Unicom has uncovered safety vulnerabilities of R1 in Chinese contexts, the safety capabilities of the remaining distilled models in the R1 series have not yet been comprehensively evaluated. To address this gap, this study utilizes the comprehensive Chinese safety benchmark CHiSafetyBench to conduct an in-depth safety evaluation of the DeepSeek-R1 series distilled models. The objective is to assess the safety capabilities of these models in Chinese contexts both before and after distillation, and to further elucidate the adverse effects of distillation on model safety. Building on these findings, we implement targeted safety enhancements for the entire DeepSeek-R1 model series. Evaluation results indicate that the enhanced models achieve significant improvements in safety while maintaining reasoning capabilities without notable degradation. We open-source the safety-enhanced models at https://github.com/UnicomAI/DeepSeek-R1-Safe to serve as a valuable resource for future research and optimization of DeepSeek models.
BibTeX
@misc{zhang2025safety,
  author = {Wenjing Zhang and Xuejiao Lei and Zhaoxiang Liu and Limin Han and Jiaojiao Zhao and Beibei Huang and Zhenhong Long and Junting Guo and Meijuan An and Rongjia Du and Ning Wang and Kai Wang and Shiguo Lian},
  title = {{Safety Evaluation and Enhancement of DeepSeek Models in Chinese Contexts}},
  year = {2025},
  eprint = {2503.16529},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2503.16529},
}
Figure