Toward a Public MAGIC Gamma-Ray Telescope Legacy Data Portal

M. Doro, Cosimo Nigro, E. Prandini, A. Tramacere, M. Delfino, J. Delgado, Elia do Souto, L. Jouvin, J. Rico

Abstract

The MAGIC telescopes are one of the three major IACTs (Imaging Atmospheric Cherenkov Telescopes) for observation of gamma rays in the TeV regime currently operative. MAGIC functions since 2003, and has published data from more than 60 sources, mostly blazars. MAGIC already provides astronomical .fits files with basic final scientific products such as spectral energy distributions, light curves and skymaps from published results. In future, the format of the files can be complemented with further relevant information to the community: a) by including the full multi-wavelength dataset enclosed in a publication, b) providing data in alternative easy-to-use formats such as ASCII or XML, which are accessible with other commonly used packages such as gammapy. Finally, besides high level products, activities have started to provide photon event lists and instrument response functions in a format such that scientists within and outside the community are allowed to perform higher level analysis. A second aim is to provide a full legacy of MAGIC data. This contribution will illustrate the achievements and plans of this activity.

Introduction: The Complexity of TeV Gamma-Ray Observation

Observing the universe in the tera-electronvolt (TeV) regime is a complex undertaking. Unlike traditional optical telescopes, the MAGIC telescopes—a pair of 17-meter dishes located in La Palma—do not capture direct images of celestial bodies. Instead, they detect faint, nanosecond-long flashes of Cherenkov light produced when high-energy gamma rays interact with the Earth's atmosphere.

Converting these flashes into usable scientific data requires a sophisticated multi-stage pipeline. Historically, sharing this data has been difficult because a simple list of detected "events" is meaningless without the corresponding Instrument Response Function (IRF). The IRF describes the telescope's sensitivity, atmospheric conditions, and hardware performance at the specific moment of observation.

The IACT data processing pipeline, illustrating the steps from raw image cleaning to final event classification.

Methods: A Tiered Approach to Data Dissemination

To address these challenges, the MAGIC Collaboration is transitioning its 15-year archive into a standardized, public legacy portal. The authors propose a tiered approach based on emerging community standards:

  • Data Level 3 (DL3): This level includes reconstructed photon event lists (detailing time, direction, and energy) paired with the necessary IRFs. By providing DL3 data, MAGIC enables researchers outside the collaboration to use open-source tools like gammapy for independent high-level analysis.
  • Data Level 4 (DL4): These are "science-ready" products, such as Spectral Energy Distributions (SEDs) and light curves. The project introduces a novel format using .yaml for metadata and .ecsv (Extended Comma Separated Values) for data tables, ensuring the files are both human-readable and machine-parseable.
  • API-First Design: The portal is designed for modern workflows, allowing users to query data via APIs (such as astroquery) rather than relying solely on manual downloads.

Case Study: Multi-Wavelength Integration of NGC 1275

A core philosophy of the new portal is that high-energy physics rarely makes sense in isolation. The MAGIC team aims to provide the entire multi-wavelength (MWL) dataset associated with a publication, rather than just the MAGIC-specific data points.

As a primary prototype, the paper highlights data from the active galaxy NGC 1275 during its significant flaring activity in 2016–2017. This dataset demonstrates the integration of MAGIC gamma-ray data with observations from Fermi-LAT (space-based gamma-rays) and the KVA telescope (optical R-band).

Multi-wavelength light curves for the NGC 1275 flaring activity, showcasing the integration of MAGIC, Fermi-LAT, and KVA datasets.

Key Insights and Future Outlook

The transition to an open data model reveals that metadata, rather than the raw data volume, is often the primary bottleneck. By standardizing metadata tags and using accessible file formats, the MAGIC Collaboration ensures that the context of an observation—such as zenith angle and sky quality—is preserved for future generations of astronomers.

While the project aligns MAGIC with the future standards of the Cherenkov Telescope Array (CTA), significant work remains. The "backlog" of 15 years of published papers must be manually processed into the new formats. However, once complete, this legacy portal will provide a robust foundation for reproducible multi-messenger astronomy, allowing the global scientific community to extract new insights from nearly two decades of high-energy observations.

Summary updated

Abstract

The MAGIC telescopes are one of the three major IACTs (Imaging Atmospheric Cherenkov Tele- scopes) for observation of gamma rays in the TeV regime currently operative. MAGIC functions since 2003, and has published data from more than 60 sources, mostly blazars. MAGIC already provides astronomical .fits files with basic final scientific products such as spectral energy distributions, light curves and skymaps from published results. In future, the format of the files can be complemented with further relevant information to the community: a) by including the full multi-wavelength dataset enclosed in a publication, b) providing data in alternative easy-to-use formats such as ASCII or XML, which are accessible with other commonly used packages such as gammapy. Finally, besides high level products, activities have started to provide photon event lists and instrument response functions in a format such that scientists within and outside the community are allowed to perform higher level analysis. A second aim is to provide a full legacy of MAGIC data. This contribution will illustrate the achievements and plans of this activity.
BibTeX
@inproceedings{doro2019toward,
  author = {M. Doro and Cosimo Nigro and E. Prandini and A. Tramacere and M. Delfino and J. Delgado and Elia do Souto and L. Jouvin and J. Rico},
  title = {{Toward a Public MAGIC Gamma-Ray Telescope Legacy Data Portal}},
  booktitle = {Proceedings of 36th International Cosmic Ray Conference — PoS(ICRC2019)},
  year = {2019},
  doi = {10.22323/1.358.0666},
}
Figure