RADiff: Controllable Diffusion Models for Radio Astronomical Maps Generation

Renato Sortino, Thomas Cecconello, Andrea DeMarco, Giuseppe Fiameni, Andrea Pilzer, Daniel Magro, Andrew M. Hopkins, Simone Riggi, Eva Sciacca, Adriano Ingallinera, Cristobal Bordiu, Filomena Bufano, Concetto Spampinato

TL;DR

This paper introduces RADiff, a novel conditional diffusion model for generating realistic radio astronomical images. RADiff addresses the critical data scarcity in radio astronomy by creating synthetic images with controllable source morphologies and backgrounds, significantly improving the performance of downstream tasks like semantic segmentation.

What Problem Does This Paper Address?

The upcoming Square Kilometre Array (SKA) will produce an unprecedented volume of radio astronomical data. Extracting valuable information, particularly through automated source detection and classification, is crucial. Deep learning models excel at these tasks but require vast amounts of meticulously labeled data. Manually annotating radio astronomical images is a time-consuming and expensive process, making it difficult to scale with the data deluge. This bottleneck limits the application of deep learning to tasks like semantic segmentation, especially when dealing with imbalanced datasets where certain object classes are underrepresented. The paper aims to overcome this data limitation by generating high-quality synthetic radio astronomical images and their corresponding annotations.

Methods

RADiff is built upon the Latent Diffusion Model (LDM) architecture, which is known for its efficiency in generating high-resolution images. The core idea is to condition the image generation process on specific inputs, allowing for controllable synthesis. RADiff uses two main conditioning mechanisms:

  1. Semantic Segmentation Masks: These masks define the shape, location, and category of the radio sources to be generated. This allows for precise control over the structure of the synthetic objects.
  2. Background Information: RADiff can also incorporate background features from a real image, enabling the transfer of realistic sky noise and diffuse emission patterns to the generated synthetic images.

The model first trains an autoencoder to compress images into a lower-dimensional latent space, reducing computational complexity. A diffusion model then operates in this latent space, learning to reverse a noise-adding process to generate new data. A conditional encoder embeds the segmentation mask and background information into a latent representation that guides the diffusion process.

Datasets

The primary dataset used is the Survey Collection (SC), which comprises cutouts (128x128 pixels) from various galactic radio astronomical surveys. These surveys were conducted using telescopes like ATCA, ASKAP, and VLA. The dataset includes images and corresponding manually annotated segmentation masks for different object classes: compact sources, extended sources, and spurious sources. For generating synthetic masks, the authors also utilized an unconditional diffusion model (DDPM) to create 5,000 synthetic masks.

Key Insights and Ideas

  • Controllable Generation is Key: The ability to control both the objects (via semantic masks) and the background (via image conditioning) is crucial for generating realistic and useful synthetic radio astronomical data.
  • LDMs for Radio Astronomy: Latent Diffusion Models prove to be a powerful and efficient tool for generating high-resolution radio astronomical images, outperforming traditional GANs and even standard diffusion models in terms of FID score.
  • Synthetic Data Solves Imbalance: Augmenting datasets with synthetic images, especially when targeting underrepresented classes, significantly improves the performance of semantic segmentation models.
  • Fully Synthetic Data Pipelines: The framework can generate both synthetic images and their corresponding masks, enabling fully automated data augmentation pipelines.

Key Results

  • Unconditional Generation: RADiff achieved a significantly lower Fréchet Inception Distance (FID) score (33.69) compared to DCGAN (101.31), PG-GAN (79.33), and DDPM (83.26) on their dataset, indicating superior generation quality.
  • Semantic Image Synthesis: RADiff, with both mask and background conditioning, achieved the best results in terms of FID (13.17), SSIM (39.04%), and Segmentation Score (60.96%), outperforming baseline models like SPADE and INADE.
  • Data Augmentation for Segmentation: Training a semantic segmentation model on datasets augmented with RADiff's synthetic images led to substantial performance improvements. Gains of up to 18% were observed when using real masks for augmentation, and up to 4% when using fully synthetic image-mask pairs. This improvement was particularly notable for addressing class imbalances.
  • Large-Scale Map Generation: The model successfully generated large-scale radio maps populated with synthetic objects, demonstrating its potential for creating realistic simulations for data challenges.

Limitations and Future Work

  • Mask Quality Dependency: The effectiveness of the synthetic image generation is limited by the quality of the segmentation masks. For optimal results, well-designed masks are essential.
  • 3D Data Extension: The authors suggest extending the pipeline to generate 3D spectral data cubes, which would be a significant advancement for radio astronomy data analysis.
  • Architectural Improvements: Further exploration into modular preprocessing functions and architectural changes to the pipeline could enhance its suitability for specific tasks, particularly for large-scale map generation.
  • Computational Cost: While LDMs are more efficient than standard diffusion models, training and inference can still be computationally intensive, especially for high-resolution images.

Summary updated

Abstract

Along with the nearing completion of the square kilometer array (SKA), comes an increasing demand for accurate and reliable automated solutions to extract valuable information from the vast amount of data it will allow acquiring. Automated source finding is a particularly important task in this context, as it enables the detection and classification of astronomical objects. Deep-learning-based object detection and semantic segmentation models have proven to be suitable for this purpose. However, training such deep networks requires a high volume of labeled data, which is not trivial to obtain in the context of radio astronomy. Since data needs to be manually labeled by experts, this process is not scalable to large dataset sizes, limiting the possibilities of leveraging deep networks to address several tasks. In this work, we propose RADiff, a generative approach based on conditional diffusion models trained over an annotated radio dataset to generate synthetic images, containing radio sources of different morphologies, to augment existing datasets and reduce the problems caused by class imbalances. We also show that it is possible to generate fully synthetic image-annotation pairs to automatically augment any annotated dataset. We evaluate the effectiveness of this approach by training a semantic segmentation model on a real dataset augmented in two ways: 1) using synthetic images obtained from real masks; and 2) generating images from synthetic semantic masks. Finally, we also show how the model can be applied to populate background noise maps for simulating radio maps for data challenges.
BibTeX
@article{sortino2024radiff,
  author = {Renato Sortino and Thomas Cecconello and Andrea DeMarco and Giuseppe Fiameni and Andrea Pilzer and Daniel Magro and Andrew M. Hopkins and Simone Riggi and Eva Sciacca and Adriano Ingallinera and Cristobal Bordiu and Filomena Bufano and Concetto Spampinato},
  title = {{RADiff: Controllable Diffusion Models for Radio Astronomical Maps Generation}},
  journal = {IEEE Trans. Artif. Intell.},
  year = {2024},
  url = {https://openreview.net/forum?id=eTnVbX785s},
}
Figure