From Domains to Instances: Dual-Granularity Data Synthesis for LLM Unlearning

Xiaoyu Xu, Minxin Du, Zitong Li, Zi Liang, Zhibiao Guo, Shiyu Zhang, Peizhao Hu, Qingqing Ye, Haibo Hu

Abstract

Although machine unlearning is essential for removing private, harmful, or copyrighted content from LLMs, current benchmarks often fail to faithfully represent the true "forgetting scope" learned by the model. We formalize two distinct unlearning granularities, domain-level and instance-level, and propose BiForget, an automated framework for synthesizing high-quality forget sets. Unlike prior work relying on external generators, BiForget exploits the target model per se to elicit data that matches its internal knowledge distribution through seed-guided and adversarial prompting. Our experiments across diverse benchmarks show that it achieves a superior balance of relevance, diversity, and efficiency. Quantitatively, in the Harry Potter domain, it improves relevance by ~20 and diversity by ~0.05 while halving the total data size compared to SOTAs. Ultimately, it facilitates more robust forgetting and better utility preservation, providing a more rigorous foundation for evaluating LLM unlearning.

Introduction

Large Language Models (LLMs) often memorize vast amounts of training data, raising significant concerns regarding privacy, copyright, and the potential disclosure of harmful information. Machine unlearning has emerged as a critical technique to remove specific knowledge without the prohibitive cost of retraining from scratch. However, existing unlearning benchmarks often suffer from a mismatch between the forget set and the model's internal knowledge, a lack of granularity, and susceptibility to surface-level forgetting.

To address these challenges, we introduce BiForget, a target-model-guided framework that automates the creation of high-quality unlearning datasets at both the domain and instance levels. By leveraging the model’s own internal knowledge through adversarial probing and seed-guided synthesis, BiForget achieves superior unlearning performance and utility preservation with significantly less data than current state-of-the-art methods.

The BiForget Framework

The core philosophy of BiForget is that the target model itself is the best authority on its own internal knowledge. The framework operates across two distinct granularities:

1. Domain-Level Synthesis

For broad topics (e.g., the Harry Potter universe), BiForget employs a two-stage process:

  • Stage I (Seed-guided): The model generates "seeds" (key concepts, facts, or characters) and subsequently constructs QA pairs and summaries around them.
  • Stage II (Adversarial Probing): To uncover deeply encoded or "hidden" knowledge, the framework utilizes jailbreaking prompts and Membership Inference Attacks (MIA). This ensures the forget set includes content that standard prompting might miss.

2. Instance-Level Synthesis

For specific facts (e.g., personal identifiable information), BiForget focuses on Information Rephrasing. It prompts the model to generate diverse semantic variants of the target statement, preventing "template overfitting" where a model might forget a specific sentence but still retain the underlying fact.

Overview of the BiForget framework showing domain-level and instance-level synthesis stages.

Efficiency and Semantic Convergence

A key innovation in BiForget is its focus on efficiency. Rather than generating massive amounts of redundant data, the framework monitors semantic diversity. By using an embedding-based stopping criterion (such as SimCSE), generation ceases once the semantic diversity of the synthetic dataset plateaus. This results in datasets that are smaller yet more "dense" with the specific information the model needs to unlearn.

Convergence of semantic similarity in domain-level synthesis, showing that diversity plateaus after a certain number of samples.

Key Results and Insights

Evaluations across benchmarks like Harry Potter, WMDP (Biosecurity/Cybersecurity), and TOFU demonstrate the effectiveness of BiForget:

  • Superior Data Quality: In the Harry Potter domain, BiForget improved relevance by ~20 points and diversity by ~0.05 while halving the total data size compared to previous synthetic methods.
  • Robust Unlearning: Models unlearned with BiForget show significantly stronger resistance to adversarial attacks, such as "Enhanced GCG" jailbreaks. This indicates that the knowledge is truly suppressed rather than just hidden.
  • Utility Preservation: BiForget-trained models maintain higher accuracy on "retain" sets (preserving general utility like MMLU) while successfully forgetting the target information.

Conclusion

BiForget provides a more rigorous foundation for LLM unlearning by aligning the forget set with the target model's internal distribution. While challenges remain—such as safety alignment conflicts where a model might refuse to generate harmful content it needs to unlearn—BiForget represents a significant step toward more robust and efficient machine unlearning.

Summary updated

Abstract

Although machine unlearning is essential for removing private, harmful, or copyrighted content from LLMs, current benchmarks often fail to faithfully represent the true "forgetting scope" learned by the model. We formalize two distinct unlearning granularities, domain-level and instance-level, and propose BiForget, an automated framework for synthesizing high-quality forget sets. Unlike prior work relying on external generators, BiForget exploits the target model per se to elicit data that matches its internal knowledge distribution through seed-guided and adversarial prompting. Our experiments across diverse benchmarks show that it achieves a superior balance of relevance, diversity, and efficiency. Quantitatively, in the Harry Potter domain, it improves relevance by ${\sim}20$ and diversity by ${\sim}$0.05 while halving the total data size compared to SOTAs. Ultimately, it facilitates more robust forgetting and better utility preservation, providing a more rigorous foundation for evaluating LLM unlearning.
BibTeX
@inproceedings{xu2026from,
  author = {Xiaoyu Xu and Minxin Du and Zitong Li and Zi Liang and Zhibiao Guo and Shiyu Zhang and Peizhao Hu and Qingqing Ye and Haibo Hu},
  title = {{From Domains to Instances: Dual-Granularity Data Synthesis for LLM Unlearning}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.1424},
}
Figure