Introduction
In observational cosmology, classifying galaxies is a monumental task. While Deep Learning (DL) models now outperform humans at identifying morphological features like spiral arms or bulges, these models often function as "black boxes," making it difficult to understand the underlying logic of their classifications. Furthermore, training these models requires massive datasets, which can be computationally expensive.
This research addresses these challenges by applying Dataset Distillation (DD)—a technique that condenses a large dataset into a tiny synthetic one (e.g., 1 or 10 images per class) while retaining enough information to train a new model effectively. The authors introduce Self-Adaptive Trajectory Matching (STM) to automate this process and generate "prototypical" galaxy images that highlight critical evolutionary features.
Methodology: Self-Adaptive Trajectory Matching (STM)
The authors' primary contribution is STM, an evolution of the Matching Training Trajectories (MTT) algorithm. MTT works by aligning the parameter-update path of a "student" model (trained on synthetic data) with that of a "teacher" model (trained on real data). However, MTT requires manual tuning of the teacher's trajectory length.
STM removes this guesswork by using a validation loss curve and a statistical hypothesis test to automatically determine when to expand the teacher's trajectory or terminate the distillation. This self-adaptive approach eliminates expensive grid searches, making DD more accessible for scientific research.
Datasets and Experimental Setup
The study utilizes three primary datasets:
- Curated Galaxy Zoo 2 (GZ2): A "high-confidence" version of the GZ2 dataset, simplified into 9 distinct classes with the top 600 most confidently labeled images per class.
- CIFAR-10 & CIFAR-100: Standard computer vision benchmarks used to validate STM's general performance against previous distillation methods.
Key Results and Insights
STM consistently outperformed the original MTT algorithm on standard benchmarks. For CIFAR-10 with 1 image per class, STM reached 47.7% accuracy, compared to 46.3% for MTT. Crucially, STM showed significantly lower variance between trials, indicating higher stability and reproducibility.
On the curated GZ2 dataset, a model trained on just 10 synthetic images per class achieved 68.4% accuracy. While lower than the 89% achieved with the full dataset, this represents a massive compression of information.
The distilled images act as "AI caricatures" or morphological summaries. For example, in spiral galaxy classes, the distillation process naturally enhances the blue tint—a tracer for young star formation. This suggests the model "learned" that star formation is a vital classification feature even without explicit instruction.
Limitations and Future Work
Despite its success, challenges remain. The original GZ2 data contains significant noise and class imbalance; distilling "raw" noisy data is a hurdle for future research. Additionally, a performance gap persists between models trained on distilled versus full datasets. Future work may explore multi-modal distillation, incorporating spectral data or redshift information to visualize how chemical compositions affect a galaxy's prototypical shape.
Reference: Guan, H., et al. (2023). Discovering Galaxy Features via Dataset Distillation. arXiv:2311.17967v1.
Summary updated