Abstract
In this paper, we propose a foreground-aware dataset distillation method that enhances patch selection in a content-adaptive manner. With the rising computational cost of training large-scale deep models, dataset distillation has emerged as a promising approach for constructing compact synthetic datasets that retain the knowledge of their large original counterparts. However, traditional optimization-based methods often suffer from high computational overhead, memory constraints, and the generation of unrealistic, noise-like images with limited architectural generalization. Recent non-optimization methods alleviate some of these issues by constructing distilled data from real image patches, but the used rigid patch selection strategies can still discard critical information about the main objects. To solve this problem, we first leverage Grounded SAM2 to identify foreground objects and compute per-image foreground occupancy, from which we derive a category-wise patch decision threshold. Guided by these thresholds, we design a dynamic patch selection strategy that, for each image, either selects the most informative patch from multiple candidates or directly resizes the full image when the foreground dominates. This dual-path mechanism preserves more key information about the main objects while reducing redundant background content. Extensive experiments on multiple benchmarks show that the proposed method consistently improves distillation performance over existing approaches, producing more informative and representative distilled datasets and enhancing robustness across different architectures and image compositions.
Introduction
Training state-of-the-art deep learning models requires massive datasets and substantial computational resources. Dataset Distillation (DD) aims to mitigate this by creating a tiny synthetic dataset—often just 10 or 50 images per class—that enables a model to reach nearly the same accuracy as if it were trained on the full original dataset.
Traditional DD methods rely on complex optimization that often produces "noise-like" images that lack semantic realism and fail to generalize across different model architectures. Recent "selection-based" methods have improved upon this by selecting real patches from original images. However, these methods typically utilize rigid, fixed-size grids. This leads to a significant challenge: if a foreground object (like a dog) occupies 90% of the frame, a fixed-size crop might only capture a patch of fur, losing the object's identity entirely. Conversely, if the object is small, a full-image resize results in a low-resolution blur. This paper presents a solution that dynamically adapts selection based on foreground content.
Methodology
The proposed method utilizes a three-stage pipeline designed to treat images adaptively based on their content.
- Foreground Recognition: The system employs Grounded SAM2 (integrating Grounding DINO and Segment Anything Model 2) to automatically segment foreground objects based on class labels. From this, a Foreground Occupancy Ratio ($R_{object}$) is calculated, representing the percentage of the image occupied by the primary subject.
- Dynamic Patch Selection: This core novelty uses a dual-path strategy:
- Path A (Sparse Foreground): If the object is relatively small ($R_{object} <$ threshold), the system samples multiple candidate patches and selects the one with the highest "realism score," determined by a pre-trained classifier.
- Path B (Dense Foreground): If the object dominates the frame, the system skips cropping and resizes the entire image to preserve the object's structural integrity.
- Distillation Synthesis: Selected patches and images are ranked and synthesized into a final distilled dataset. A "soft labeling" strategy is used, where a teacher model provides nuanced labels for the synthesized images, enhancing the information density of the compact set.
Key Insights and Analysis
Category-Specific Thresholds
A critical finding of this research is that a global threshold for cropping is insufficient. Different categories exhibit different typical occupancy levels; for instance, "gas pumps" are often small in the frame, while "fish" are typically large. The authors derived thresholds based on the 30% quantile of each category's occupancy distribution to ensure the selection logic is tailored to the specific class.
The "Completeness" vs. "Detail" Trade-off
Previous selection methods prioritized high-resolution details (patches) at the expense of object completeness. This work argues that for distilled data to be effective, preserving the whole object structure is vital when that structure is available. By maintaining semantic integrity, the distilled images remain recognizable and useful across various model architectures (e.g., training on a ConvNet and testing on a ResNet).
Experimental Results
The method was evaluated on standard benchmarks including CIFAR-10/100, ImageNette, and ImageWoof. It consistently outperformed existing state-of-the-art (SOTA) selection-based methods like RDED:
- ImageNette (IPC 50): Achieved 89.5% accuracy, a significant improvement over RDED's 80.4%.
- ImageWoof (IPC 50): Reached 75.6% accuracy, compared to 68.5% for the previous SOTA.
- Low-Data Regime (IPC 1): The method showed strong gains even with only one image per class, hitting 39.5% on ImageNette while competitors remained in the 20-30% range.
Conclusion and Future Work
By introducing foreground awareness into the distillation process, this method produces synthetic datasets that are both more realistic and more effective for training. While the current approach uses a simple 2x2 grid for synthesis, future work may explore more flexible layout strategies or hybrid approaches that combine selection-based realism with optimization-based refinement to further enhance feature representation.
Summary updated