Discovering features in gravitational-wave data through detector characterization, citizen science and machine learning

S. Soni, C. P. L. Berry, S. B. Coughlin, Mahboobeh Harandi, Corey Jackson, Kevin Crowston, Carsten Østerlund, O. Patane, Aggelos K. Katsaggelos, Laura Trouille, Victor-Georges Baranowski, Wilfried Domainko, Kamil Kamiński, M A Lobato Rodriguez, U Marciniak, P Nauta, G Niklasch, Roshni Rote, B Téglás, C.p. Unsworth, C ZhangClassical 2021 · DOI

TL;DR

This paper details how citizen scientists, using the Gravity Spy platform, collaborated with LIGO experts to identify and classify new types of transient noise (glitches) in gravitational-wave data from the O3 observing run. By combining expert knowledge with volunteer insights and machine learning, they discovered two new glitch classes, "Fast Scattering/Crown" and "Low-frequency Blips," significantly improving our understanding of detector noise and paving the way for better gravitational-wave detection.

What Problem Does This Paper Address?

Detecting faint gravitational-wave signals is challenging because the data is contaminated by transient noise, or "glitches." These glitches can mimic real signals or obscure them, making it difficult to pinpoint astrophysical events. While LIGO detectors are highly sophisticated, understanding and mitigating these glitches is crucial for improving data quality and maximizing the scientific return from observations. Traditional methods of identifying glitches rely on expert analysis, which is time-consuming and can struggle with the sheer volume of data. This paper addresses the need for more efficient and comprehensive glitch identification, particularly for discovering novel glitch types that may arise with detector upgrades or changes in environmental conditions.

Methods

The study leverages a combination of expert-driven detector characterization and citizen science, powered by machine learning.

  • Expert-driven Detector Characterization: LIGO scientists analyze detector data and environmental monitoring information to identify and classify glitches. This involves using specialized software tools and examining time-frequency spectrograms.
  • Citizen Science (Gravity Spy): The Gravity Spy platform on Zooniverse enlists volunteers to classify gravitational-wave data. Volunteers are trained through a tiered system, starting with confidently classified glitches and progressing to more ambiguous ones. A key feature is a "similarity search" tool that helps identify glitches morphologically similar to a given example.
  • Machine Learning (Convolutional Neural Network - CNN): A CNN is trained on labeled glitch spectrograms to classify new data. The input to the CNN is a composite of spectrograms at different durations (0.5s, 1.0s, 2.0s, 4.0s) to capture features across various glitch timescales. The CNN outputs a probability score for each glitch class.
  • New Training Sets: The core of this work involved creating updated training sets for the CNN. This was done in two ways:
  • Expert Training Set: Assembled by LIGO experts, incorporating knowledge of detector status and identifying two new classes: Fast Scattering and Low-frequency Blip.
  • Volunteer Training Set: Developed by citizen scientists through their investigations, identifying a new class called Crown.

Datasets

The primary dataset used is from the third observing run (O3) of the Advanced LIGO detectors, covering data from April 2019 to March 2020.

  • Training Data:
    • Expert Training Set: Included new classes: Fast Scattering and Low-frequency Blip.
  • Volunteer Training Set: Included the new class: Crown.
  • The paper mentions a previous training set of 7932 images across 22 classes used for earlier observing runs.
  • The updated O3 training data set contains 9631 examples distributed over 23 classes for the Expert model.
  • The Volunteer O3 training data set contains 9496 examples distributed over 23 classes.
  • Evaluation Data: 20% of the O3 data from both Hanford and Livingston observatories was reclassified using the updated models.

Key Insights and Ideas

  • Citizen Science as a Discovery Engine: The study highlights that citizen scientists, even without direct access to auxiliary detector information, can independently discover new and significant glitch classes, mirroring findings from expert analysis. This underscores the power of crowdsourcing for scientific discovery in complex datasets.
  • Ground Motion as a Key Glitch Origin: The "Fast Scattering/Crown" glitch class is strongly correlated with ground motion at the detector sites. This connection is a significant step towards understanding and mitigating this prevalent noise source.
  • Subclasses of Fast Scattering: Two distinct subclasses of Fast Scattering were identified, each linked to different types of ground motion: one to microseismic activity (0.1-0.3 Hz) and another to anthropogenic activity (1-6 Hz).
  • Complementary Nature of Expert and Volunteer Efforts: The independent discovery of similar glitch types (Fast Scattering by experts, Crown by volunteers) demonstrates the value of both approaches. The slight differences in their findings are attributed to variations in training data and access to auxiliary information.
  • Importance of Training Data: The differences in classification between the expert and volunteer models are largely explained by the composition and characteristics of their respective training sets, particularly regarding signal-to-noise ratio (SNR) distributions.

Key Results

  • New Glitch Classes Identified: "Fast Scattering/Crown" and "Low-frequency Blips" were identified as new, prevalent glitch classes in the O3 data.
  • Prevalence of Fast Scattering/Crown: After reclassification, approximately 27% of transient noise at LIGO Livingston was identified as Fast Scattering, making it the most frequent transient noise source at that site.
  • Prevalence of Low-frequency Blips: Approximately 8% of transient noise at LIGO Livingston was identified as Low-frequency Blips, making it the fourth most frequent.
  • Model Performance: The updated machine learning models achieved high accuracies (99.9% and 98.8% for training/validation in the Expert model; 99.2% and 97.5% for the Volunteer model).
  • Reclassification Impact: At Livingston, the new model assigned a new label to 42% of the analyzed triggers, while at Hanford, this was 11%. This difference is attributed to the higher prevalence of Fast Scattering at Livingston.
  • Consistency: The Expert model classified nearly all Crown glitches as Fast Scattering (>99%), and the Volunteer model classified the majority of Fast Scattering glitches as Crown (~85%).

Limitations and Future Work

  • Data Access Differences: Volunteers lacked direct access to auxiliary information about detector states and environmental conditions, which may have influenced their classifications compared to experts.
  • Training Set Differences: The composition of the expert and volunteer training sets (e.g., SNR ranges, data sources like ER14 vs. O3) led to some differences in classification outcomes.
  • Future Work:
    • Tailoring separate machine-learning algorithms for each detector (Livingston and Hanford) due to their differing configurations and environments.
  • Further investigation into the physical origins of glitches, particularly the Low-frequency Blips, whose cause remains a mystery.
  • Continued use of citizen science and machine learning for identifying new glitch classes as detectors are upgraded and sensitivity increases.
  • The paper suggests that observatory-specific algorithms built upon the existing Gravity Spy framework could be developed using transfer learning.
  • Scientists are planning to damp ground motion and re-evaluate its impact on detector data quality.

Summary updated

Abstract

Abstract The observation of gravitational waves is hindered by the presence of transient noise (glitches). We study data from the third observing run of the Advanced LIGO detectors, and identify new glitch classes: fast scattering/crown and low-frequency blips . Using training sets assembled by monitoring of the state of the detector, and by citizen-science volunteers, we update the Gravity Spy machine-learning algorithm for glitch classification. We find that fast scattering/crown, linked to ground motion at the detector sites, is especially prevalent, and identify two subclasses linked to different types of ground motion. Reclassification of data based on the updated model finds that ∼27% of all transient noise at LIGO Livingston belongs to the fast scattering class, while ∼8% belongs to the low-frequency blip class, making them the most frequent and fourth most frequent sources of transient noise at that site. Our results demonstrate both how glitch classification can reveal potential improvements to gravitational-wave detectors, and how, given an appropriate framework, citizen-science volunteers may make discoveries in large data sets.
BibTeX
@inproceedings{soni2021discovering,
  author = {S. Soni and C. P. L. Berry and S. B. Coughlin and Mahboobeh Harandi and Corey Jackson and Kevin Crowston and Carsten Østerlund and O. Patane and Aggelos K. Katsaggelos and Laura Trouille and Victor-Georges Baranowski and Wilfried Domainko and Kamil Kamiński and M A Lobato Rodriguez and U Marciniak and P Nauta and G Niklasch and Roshni Rote and B Téglás and C.P. Unsworth and C Zhang},
  title = {{Discovering features in gravitational-wave data through detector characterization, citizen science and machine learning}},
  booktitle = {Classical},
  year = {2021},
  doi = {10.1088/1361-6382/ac1ccb},
}
Figure