SEARCHING FOR PULSARS USING IMAGE PATTERN RECOGNITION

Weiwei Zhu, A. Berndsen, E. C. Madsen, Maxine Tan, I. H. Stairs, Adam Brazier, P. Lazarus, Ryan S. Lynch, Paul Scholz, K. Stovall, S. M. Ransom, S. Banaszak, Christopher M. Biwer, S L Cohen, L. P. Dartez, Jeffrey Flanigan, G. Lunsford, José Martinez, A. Mata, M. Rohr, A. Walker, B. Allen, N. D. R. Bhat, Slavko Bogdanov, F. Camilo, Shami Chatterjee, J. M. Cordes, F. Crawford, J. S. Deneva, G. Desvignes, R. D. Ferdman, P. C. C. Freire, J. W. T. Hessels, F. A. Jenet, D. L. Kaplan, V. M. Kaspi, Benjamin Knispel, K. J. Lee, J. van Leeuwen, A. G. Lyne, M. A. McLaughlin, Xavier Siemens, L. G. Spitler, A. VenkataramanThe Astrophysical Journal 2014 · DOI

TL;DR

This paper introduces PICS, a novel AI system that uses deep neural networks and image pattern recognition to identify pulsars from astronomical survey data. PICS achieves high accuracy by learning from human-labeled pulsar candidates, outperforming traditional methods and enabling faster, more efficient pulsar discovery.

What Problem Does This Paper Address?

Modern astronomical surveys are generating vast amounts of data, creating a bottleneck in the analysis phase. Specifically, identifying pulsars from millions of potential candidates is a time-consuming and labor-intensive task, even for human experts. Existing automated methods often rely on pre-defined heuristics that can miss subtle or unusual pulsar signals and struggle with radio frequency interference (RFI). This paper aims to bridge this gap by developing an AI that can effectively and efficiently sift through these massive datasets, mimicking human expert analysis but at a much larger scale.

Methods

The core of the PICS system is a two-layer artificial intelligence (AI) architecture.

  • First Layer: Feature Specialists: This layer employs a combination of machine learning algorithms (Artificial Neural Networks - ANN, Convolutional Neural Networks - CNN, and Support Vector Machines - SVM) to analyze four key diagnostic plots generated for each pulsar candidate. These plots represent the pulse profile, time-versus-phase, frequency-versus-phase, and DM curve. Each specialist classifier is trained to rate how "pulsar-like" a particular subplot is, outputting a prediction matrix.
  • Second Layer: Consensus Builder: A logistic regression (LR) classifier takes the outputs from the first layer specialists and learns to weight them appropriately. This layer then combines these weighted scores to produce a final consensus score, indicating the overall probability of a candidate being a pulsar.

Key novelties include:

  • Image Pattern Recognition: PICS directly uses image patterns from diagnostic plots, rather than relying solely on pre-defined heuristic scores.
  • Deep Neural Networks: The use of deep neural networks allows PICS to learn complex and subtle features that might be missed by simpler algorithms.
  • Two-Layer Hierarchy: This "committee of experts" approach allows for specialized analysis of different features and a robust final decision.
  • Data Preparation: Techniques like down-sampling, interpolation, normalization, and optional PCA are used to ensure consistent input data for the ML models.

Datasets

  • PALFA Survey Data: Used for training the PICS AI system. This dataset consists of human-labeled pulsar candidates, including pulsars, their harmonics, and non-pulsars (RFI).
  • Green Bank North Celestial Cap (GBNCC) Survey Data: Used for independent validation of the trained PICS AI. This dataset is distinct from the training data and represents a new benchmark for evaluating the AI's generalization capabilities.

Key Insights and Ideas

  • Mimicking Human Expertise: The AI is designed to emulate how human experts visually inspect diagnostic plots, capturing the nuanced patterns that indicate a pulsar.
  • Harmonics are Crucial: PICS is sensitive to pulsar harmonics, which can be important for identifying pulsars that might be missed by traditional methods due to RFI affecting the fundamental frequency.
  • Focus on Image Patterns: By focusing on the visual features within the diagnostic plots and ignoring other parameters like period or DM, PICS can act as a valuable complement to existing score-based systems.
  • RFI Rejection Power: The AI demonstrates a strong ability to reject RFI, even complex signals that can mimic pulsar characteristics.
  • Generalizability: The AI's performance on the independent GBNCC dataset suggests its applicability to other pulsar surveys with minimal modification.

Key Results

  • High Classification Accuracy: On the independent GBNCC test set, PICS ranked 100% of known pulsars and 94% of their harmonics within the top 1% of all candidates.
  • Effective RFI Rejection: PICS rejected approximately 96.2% of 60 Hz RFI signals, with false positives strongly resembling pulsars.
  • Improved Efficiency: By ranking candidates, PICS can improve the speed of candidate classification for human experts by a factor of at least 100.
  • New Discoveries: The PICS AI has been integrated into the PALFA survey pipeline and has already led to the discovery of six new pulsars.
  • Performance Metrics: Individual first-layer classifiers achieved F1 scores in the range of 86-94%, while the overall PICS system achieved an average F1 score of 96% on validation data.

Limitations and Future Work

  • Training Data Diversity: The authors acknowledge that the quantity and diversity of the training data are limitations. Expanding the training set with more pulsar/RFI candidates, especially those with broad pulse profiles, could further improve performance.
  • Scintillation and Binary Motion: The current AI is trained on data with short exposure times, where scintillation and binary motion effects are minimal. For surveys with longer exposure times, the AI would need to be retrained with candidates exhibiting these effects, or potentially with simulated data.
  • Online Learning: The authors envision a future system that can adaptively train the AI on the fly as new data and RFI environments emerge, ensuring portability and continuous improvement.
  • Imbalanced Data: While the AI performs well, dealing with the inherent imbalance between the vast number of RFI candidates and the relatively few pulsars remains an ongoing challenge.

Summary updated

Abstract

In the modern era of big data, many fields of astronomy are generating huge volumes of data, the analysis of which can sometimes be the limiting factor in research. Fortunately, computer scientists have developed powerful data-mining techniques that can be applied to various fields. In this paper, we present a novel artificial intelligence (AI) program that identifies pulsars from recent surveys by using image pattern recognition with deep neural nets—the PICS (Pulsar Image-based Classification System) AI. The AI mimics human experts and distinguishes pulsars from noise and interference by looking for patterns from candidate plots. Different from other pulsar selection programs that search for expected patterns, the PICS AI is taught the salient features of different pulsars from a set of human-labeled candidates through machine learning. The training candidates are collected from the Pulsar Arecibo L-band Feed Array (PALFA) survey. The information from each pulsar candidate is synthesized in four diagnostic plots, which consist of image data with up to thousands of pixels. The AI takes these data from each candidate as its input and uses thousands of such candidates to train its ~9000 neurons. The deep neural networks in this AI system grant it superior ability to recognize various types of pulsars as well as their harmonic signals. The trained AI's performance has been validated with a large set of candidates from a different pulsar survey, the Green Bank North Celestial Cap survey. In this completely independent test, the PICS ranked 264 out of 277 pulsar-related candidates, including all 56 previously known pulsars and 208 of their harmonics, in the top 961 (1%) of 90,008 test candidates, missing only 13 harmonics. The first non-pulsar candidate appears at rank 187, following 45 pulsars and 141 harmonics. In other words, 100% of the pulsars were ranked in the top 1% of all candidates, while 80% were ranked higher than any noise or interference. The performance of this system can be improved over time as more training data are accumulated. This AI system has been integrated into the PALFA survey pipeline and has discovered six new pulsars to date.
BibTeX
@article{zhu2014searching,
  author = {Weiwei Zhu and A. Berndsen and E. C. Madsen and Maxine Tan and I. H. Stairs and Adam Brazier and P. Lazarus and Ryan S. Lynch and Paul Scholz and K. Stovall and S. M. Ransom and S. Banaszak and Christopher M. Biwer and S L Cohen and L. P. Dartez and Jeffrey Flanigan and G. Lunsford and José Martinez and A. Mata and M. Rohr and A. Walker and B. Allen and N. D. R. Bhat and Slavko Bogdanov and F. Camilo and Shami Chatterjee and J. M. Cordes and F. Crawford and J. S. Deneva and G. Desvignes and R. D. Ferdman and P. C. C. Freire and J. W. T. Hessels and F. A. Jenet and D. L. Kaplan and V. M. Kaspi and Benjamin Knispel and K. J. Lee and J. van Leeuwen and A. G. Lyne and M. A. McLaughlin and Xavier Siemens and L. G. Spitler and A. Venkataraman},
  title = {{SEARCHING FOR PULSARS USING IMAGE PATTERN RECOGNITION}},
  journal = {The Astrophysical Journal},
  year = {2014},
  doi = {10.1088/0004-637x/781/2/117},
}
Figure