Learning and interpreting gravitational-wave features from CNNs with a random forest approach

Jun Tian, He Wang, J. He, Yu Pan, Shuo Cao, Qing-Quan Jiang

TL;DR

This paper introduces a novel hybrid CNN-Random Forest (RF) model for gravitational wave (GW) detection that significantly improves both detection sensitivity and interpretability. By combining a deep learning feature extractor with physically motivated handcrafted features, the model achieves a 21% increase in sensitivity at a low false alarm rate and offers better insights into the detection process.

What Problem Does This Paper Address?

Gravitational wave (GW) astronomy has seen a surge in detections thanks to advanced observatories like LIGO and Virgo. While Convolutional Neural Networks (CNNs) have shown great promise in automatically learning complex features from GW data, their "black box" nature makes it difficult to understand why they make certain predictions. This lack of interpretability hinders trust and limits our ability to diagnose errors or improve models. Existing methods often directly feed CNN outputs to a classifier, missing opportunities to incorporate domain knowledge. This paper aims to bridge this gap by developing a more interpretable and performant GW detection system.

Methods

The core of this work is a hybrid CNN-Random Forest (RF) architecture. First, a pre-trained CNN (trained on GW signals and noise) acts as a feature extractor. Crucially, instead of just using the CNN's final output probability, the researchers extract four physically interpretable handcrafted features from the CNN's final convolutional layer:

  • Variance: Measures the spread of data points around the mean, indicating statistical distribution.
  • Signal-to-Noise Ratio (SNR): Adapted from matched filtering, it quantifies coherent amplitude consistency between detector signals.
  • Waveform Overlap: Also inspired by matched filtering, it measures the phase similarity between detector signals.
  • Peak Amplitude: Identifies the maximum value in the feature maps, reflecting signal strength.

These four handcrafted features, along with the CNN's output probability, are then fed into an RF classifier. The RF's ability to provide feature importance scores is key to understanding which features contribute most to the detection decision.

Datasets

The study utilized three datasets:

  • Dataset 1: Synthetic GW signals and pure noise, generated using the ggwd project. This dataset was used for initial CNN training.
  • Dataset 2: Synthetic GW signals, pure noise, and glitches (transient noise artifacts). This dataset was used for training the RF classifier and for feature analysis.
  • Dataset 3: Long-duration time-series data (one week of O1 run data) with randomly injected GW signals. This dataset was used for evaluating model performance in a more realistic scenario.

Key Insights and Ideas

  • Handcrafted features are powerful: The study demonstrates that incorporating physically motivated features (like SNR and variance) derived from CNN outputs significantly boosts detection performance and interpretability. These features capture crucial aspects of GW signals that might be missed by a purely data-driven approach.
  • Interpretability enhances performance: By making the detection process more transparent, the hybrid model not only improves performance but also allows for better understanding of why certain signals are detected or missed.
  • CNNs learn more than just GWs: Even when trained only on GWs and noise, the baseline CNN showed an ability to extract useful information about glitches, which is crucial for real-world data.
  • RF's feature importance is key: The RF classifier's ability to rank feature importance revealed that handcrafted features, particularly variance, were highly discriminative, especially for distinguishing glitches from GW signals.

Key Results

  • Improved Sensitivity: The proposed CNN-RF model achieved a 21% relative improvement in sensitivity compared to the baseline-CNN at a false alarm rate (FAR) of 10 events per month.
  • Low-SNR Detection: The hybrid model showed improved detection of low-SNR signals (SNR < 10), which are often challenging.
  • High AUC Scores: The CNN-RF model achieved consistently high Area Under the Curve (AUC) scores (≥ 0.95) for classifying GW signals, noise, and glitches, indicating robust performance across all classes.
  • Feature Dominance: Feature importance analysis showed that the combined handcrafted features accounted for 74.17% of the decision-making process, with variance being the most important feature.
  • Inference Time Trade-off: The hybrid model requires approximately four times more inference time than the baseline-CNN, which is a trade-off for improved performance and interpretability.

Limitations and Future Work

  • Computational Overhead: The inclusion of handcrafted features and the RF classifier increases computational cost, particularly for inference time, which could limit its applicability in real-time, low-latency pipelines.
  • Signal Type Focus: The current study primarily focuses on Binary Black Hole (BBH) signals and a two-detector configuration.
  • Future Directions: The authors plan to extend the framework to:
  • Accommodate a broader range of astrophysical sources (e.g., core-collapse supernovae, LISA signals).
  • Incorporate data from multi-detector networks (Virgo, KAGRA, LIGO-India) for enhanced robustness and generalizability.
  • Explore more flexible, data-driven feature selection strategies that maintain physical interpretability.

Summary updated

Abstract

Abstract Convolutional neural networks (CNNs) have attracted increasing attention in gravitational wave (GW) data analysis due to their ability to automatically learn hierarchical features from raw strain data. However, the physical meaning of these learned features remains underexplored, limiting the interpretability of such models. In this work, we propose a hybrid architecture that combines a CNN-based feature extractor with a random forest (RF) classifier to improve both detection performance and interpretability. Unlike prior approaches that directly connect classifiers to CNN outputs, our method introduces four physically interpretable metrics-variance, signal-to-noise ratio (SNR), waveform overlap, and peak amplitude-computed from the final convolutional layer. These are jointly used with the CNN output probability in the RF classifier to enable more informed decision boundaries. Tested on long-duration strain datasets, our hybrid model outperforms a baseline-CNN model, achieving a relative improvement of 21% in sensitivity at a fixed false alarm rate of 10 events per month. Notably, it also shows improved detection of low-SNR signals (SNR <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" overflow="scroll"> <mml:mrow> <mml:mtext>⩽</mml:mtext> </mml:mrow> </mml:math> 10), which are especially vulnerable to misclassification in noisy environments. Feature importance analysis and ablation studies reveal that handcrafted features play a significant role in classification decisions and contribute to improved performance. These findings suggest that physically motivated post-processing of CNN feature maps can serve as a valuable tool for interpretable and efficient GW detection, bridging the gap between deep learning and domain knowledge.
BibTeX
@inproceedings{tian2025learning,
  author = {Jun Tian and He Wang and J. He and Yu Pan and Shuo Cao and Qing-Quan Jiang},
  title = {{Learning and interpreting gravitational-wave features from CNNs with a random forest approach}},
  booktitle = {Machine},
  year = {2025},
  doi = {10.1088/2632-2153/adfc27},
}
Figure