TL;DR
Researchers have successfully applied self-supervised contrastive learning to "in the wild" radio astronomy data, creating a model that learns to categorize galaxies without needing a single human-provided label. By carefully filtering out small, unresolved sources, the team achieved classification performance that rivals supervised models while enabling powerful new capabilities like similarity searches for rare cosmic objects.
What Problem Does This Paper Address?
Radio astronomy is entering the era of "Big Data." Next-generation telescopes like the Square Kilometre Array (SKA) will produce exabytes of data, revealing millions of new radio galaxies. Traditionally, astronomers classify these galaxies into categories like Fanaroff-Riley I and II (FRI/FRII) based on their shapes.
However, supervised deep learning—the current gold standard—requires massive amounts of labeled data. Labeling is expensive, slow, and prone to human bias. While semi-supervised learning (using a mix of labeled and unlabeled data) seems like a solution, it often breaks down when the unlabeled "wild" data has a different distribution than the labeled training set—a phenomenon known as dataset shift. This paper explores whether self-supervised representation learning can bridge this gap by learning the underlying structure of the data itself, without needing labels upfront.
Methods
The authors utilized a contrastive learning framework called BYOL (Bootstrap Your Own Latent). Unlike other popular methods (like SimCLR), BYOL does not require "negative pairs" (explicitly telling the model which images are different). Instead, it uses two neural networks—an "online" and a "target" network—that learn from different augmented views of the same image.
Key technical choices included:
- Backbone: A ResNet-18 architecture.
- Augmentation Tweaks: The authors found that standard computer vision augmentations (like heavy blurring) destroyed the fine-grained morphological details of radio galaxies. They reduced the blurring kernel size and probability to preserve scientific information.
- Data Curation: The most critical step was implementing an arcsecond cut threshold. They removed sources with low angular extent (small, unresolved dots) because these sources lack the morphological features the model needs to learn useful representations.
Datasets
The study utilized two primary datasets:
- MiraBest: A high-quality, human-labeled dataset of 1,256 radio galaxies used for evaluation and as a supervised baseline.
- Radio Galaxy Zoo Data Release 1 (RGZ DR1): A much larger, "wild" dataset containing ~100,000 unlabeled sources from the FIRST survey. This served as the primary training set for the self-supervised model.
Key Insights and Ideas
- Curation is King: The most striking insight is that "more data" isn't always better. Including small, unresolved sources in the training set caused the model's performance to plateau. By removing sources smaller than ~25 arcseconds, the model was forced to focus on complex shapes, leading to a "qualitatively different training dynamic."
- Closing the Gap: Even though the model never saw a label during training, a simple k-nearest neighbor (kNN) classifier applied to its learned features could separate FRI and FRII galaxies with high accuracy.
- Beyond Classification: Because the model learns a general "representation space," it can be used for tasks other than classification. The authors demonstrated this via similarity searches, where the model could find rare "hybrid" galaxies or even identify duplicate entries in the catalog that had escaped previous automated filters.
Key Results
- Accuracy: The self-supervised model achieved a classification accuracy of 85.25% on the MiraBest test set. While lower than the supervised baseline (92.42%), it is remarkably close considering the self-supervised model used zero labels and was trained on a completely different dataset (RGZ DR1).
- The Threshold Effect: Performance was highly sensitive to the angular extent cut. At low thresholds (including small sources), validation accuracy remained flat. As the threshold increased toward 25–30 arcseconds, accuracy improved by over 10%.
- Discovery: The model successfully retrieved semantically similar "hybrid" sources (objects sharing both FRI and FRII traits) and identified duplicates in the RGZ DR1 catalog that were slightly offset or cropped differently.
Limitations and Future Work
The authors acknowledge that their current data curation (the arcsecond cut) relies on domain-specific knowledge. A major goal for future work is to find domain-agnostic ways to automatically filter or weight data so that models can learn effectively from even noisier "wild" datasets.
Additionally, while this work focuses on the FRI/II distinction, the ultimate goal is to develop "foundation models" for astronomy—large-scale models that can be fine-tuned for a variety of downstream tasks, from identifying black hole jets to discovering entirely new classes of astronomical phenomena.
Summary updated