DR-DPO: Dual-Regularized DPO for Efficient Dataset Condensation

Haiduo Huang, Jiangcheng Song, Yadong Zhang, Guansu Wang, Pengju RenCVPR 2026 Findings 2026 · OpenReview

TL;DR

DR-DPO reimagines dataset condensation as a preference optimization problem, using a dual-regularized framework to simultaneously maximize class separability and preserve intra-class diversity. By replacing expensive bilevel optimization with a clean, single-level $O(B)$ objective, it beats state-of-the-art methods by up to 5% in accuracy while slashing peak memory usage by up to 10.2× and training time by up to 7.1×.

What Problem Does This Paper Address?

Deep learning relies on massive datasets, making model training computationally expensive and resource-heavy. Dataset condensation aims to solve this by synthesizing a tiny, highly informative set of synthetic images that can train a model to nearly the same accuracy as the full dataset.

However, existing approaches suffer from major bottlenecks. Traditional gradient- and distribution-matching methods ignore explicit class relationships, leading to weak decision boundaries and poor discriminative power. On the other hand, advanced trajectory-matching methods require expensive bilevel optimization (unrolling training loops) that quickly exhausts GPU memory. The authors tackle this dual challenge: how to explicitly optimize both inter-class distinctiveness and intra-class diversity without incurring massive computational overhead.

Methods

The authors introduce DR-DPO (Dual-Regularized Direct Preference Optimization), which frames dataset condensation through the lens of preference learning (popularized by LLM alignment methods like DPO).

The framework consists of two core components optimized jointly in a single forward pass:

  1. Inter-Class Preference Loss via DPO: Instead of using contrastive losses that require expensive $O(B^2)$ pair mining, DR-DPO uses a log-ratio preference objective. It treats the correct class as the "preferred" option and a randomly sampled incorrect class as the "rejected" option, maximizing the relative margin between synthetic and real data with an efficient $O(B)$ complexity.
  2. Intra-Class Diversity Loss: To prevent "prototype collapse" (where all synthetic images in a class look identical), the authors introduce a regularizer based on Jensen-Shannon (JS) divergence. It ensures that the diversity between pairs of synthetic samples within the same class meets or exceeds their initial state.
Figure 2. Overview of the DR-DPO framework. (1) Inter-class preference-inspired loss maximizes separability via log-ratio optimization; (2) Intra-class JS regularization preserves within-class diversity.

Datasets

The method was evaluated on standard computer vision benchmarks:

  • CIFAR-10 and CIFAR-100
  • Tiny-ImageNet
  • Six ImageNet-1K subsets (ImageNette, ImageWoof, ImageFruit, ImageMeow, ImageSquawk, ImageYellow) scaled to 128×128 resolution to evaluate scalability and performance under class imbalance.

No new datasets were introduced in this paper.

Key Insights and Ideas

  • DPO Translates Beautifully to Vision: Direct Preference Optimization isn't just for language models. By framing a synthetic sample's class assignment as a preference over real data margins, the mathematical properties of DPO (like bounded gradients and shift-invariance) provide highly stable optimization for image synthesis.
  • Bilevel Optimization is Avoidable: You don't need complex min-max games or unrolled training trajectories to condense data effectively. A well-designed single-level objective can achieve superior accuracy with a fraction of the compute.
  • Diversity Needs an Anchor: Simply maximizing intra-class variance can lead to over-dispersion or chaotic images. Calibrating the JS divergence against the initialization state acts as a perfect baseline to maintain natural intra-class variability without excessive spread.

Key Results

  • State-of-the-Art Accuracy: DR-DPO consistently outperforms strong baselines like NCFM across all configurations. It achieves top-1 accuracy gains of up to 1.8% on CIFAR-100 and up to 5.0% relative improvement on Tiny-ImageNet.
  • Massive Efficiency Gains: On a single NVIDIA A100 GPU, DR-DPO cuts training time per iteration by 1.3× to 7.2× and slashes peak memory by 4.3× to 10.2× compared to trajectory-matching and min-max baselines.
  • Cross-Architecture Robustness: Synthetic datasets condensed on a standard ConvNet successfully transfer to AlexNet, VGG-11, and ResNet-18, narrowing the generalization gap typically seen in prior dataset distillation work.
Accuracy-efficiency Pareto frontier on CIFAR-100

Limitations and Future Work

  • Hyperparameter Sensitivity: While robust within a reasonable band, DR-DPO still exhibits sensitivity to its temperature ($\beta$) and regularization weight ($\lambda$) parameters under extreme data scarcity or severe class imbalance.
  • Initialization Overhead: The multi-sample composition initialization strategy used to kickstart the process adds an upfront computational cost.
  • Future Directions: The authors suggest exploring lighter initialization methods, adaptive hyperparameter schedules, and extending the framework to non-vision modalities (like text or audio) and richer preference modeling structures.

Summary updated

Abstract

Dataset condensation aims to synthesize a tiny training set that preserves the performance of large datasets. Existing gradient- or distribution-matching pipelines either incur high computation/memory or lack explicit control over class relations. We propose DR-DPO, a dual-regularized framework that recasts condensation as preference optimization. An inter-class Direct Preference Optimization (DPO) loss maximizes class separability by operating on log-odds differences, eliminating metric/margin tuning and O(B^2) pair mining (with batch size B). An intra-class Jensen--Shannon term preserves within-class statistics to prevent prototype collapse. The resulting O(B) objective yields stable, calibrated gradients and transfers well across architectures. On CIFAR-10/100, Tiny-ImageNet, and ImageNet-1K subsets, DR-DPO sets a new state of the art, improving top-1 accuracy by up to 1.8--5.0% over strong baselines while reducing peak memory by up to 10.2× and training time by up to 7.1×.
BibTeX
@inproceedings{huang2026dr,
  author = {Haiduo Huang and Jiangcheng Song and Yadong Zhang and Guansu Wang and Pengju Ren},
  title = {{DR-DPO: Dual-Regularized DPO for Efficient Dataset Condensation}},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year = {2026},
  url = {https://openreview.net/forum?id=GmscB4v3W0},
}
Figure