Introduction: The Challenge of Background Noise in NICER
The Neutron star Interior Composition Explorer (NICER), located on the International Space Station (ISS), is a critical instrument for studying high-energy astrophysical phenomena like pulsars and black holes. However, NICER operates as a single-pixel detector, meaning it lacks the spatial resolution of CCD-based telescopes to distinguish between a target source and the surrounding background.
The resulting data stream is heavily contaminated by "background spectra"—noise generated by cosmic rays, solar activity, and the magnetic environment of the ISS. This noise is highly variable and can frequently obscure faint astrophysical signals. Traditional models, such as SCORPEON or 3C50, rely on rule-based systems or a limited set of variables, often failing to accurately predict high-intensity background "spikes."
The BERTground Architecture
To address these limitations, researchers developed BERTground, a deep-learning architecture that adapts Transformer-based models for tabular space-weather data. The model employs a three-stage pipeline to process 46 raw numerical variables describing the spacecraft's environment.
- Feature Group Tokenizers: Raw inputs (e.g., ISS longitude, magnetic field strength) are grouped into five physically relevant categories, such as "Geomagnetic" and "Solar Position." These groups are processed through sigmoid-activated linear layers to create "tokens" representing physical concepts.
- Transformer Encoder: Four Transformer layers with eight attention heads analyze the interactions between these feature groups, capturing complex non-linear relationships.
- Tabular Deep Residual Network (ResNet): Instead of a standard Multi-Layer Perceptron (MLP), a ResNet with skip connections is used for the final prediction. This architecture helps retain raw feature information and ensures stable training.
The model predicts photon counts across 1,180 energy bins (0.2 to 12 keV) using a custom Mean Squared Log Error (MSLE) loss function to maintain sensitivity across all energy levels.
Data and Training
BERTground was trained on over 3 million seconds of "blank sky" observations—regions of space known to be devoid of X-ray sources. The dataset included 46 "MKF" parameters detailing the spacecraft's environment and detector settings. Data from five regions were used for training and validation, while two additional regions were reserved for testing to evaluate the model's ability to generalize to unseen parts of the sky.
Results and Performance
BERTground represents a significant advancement over existing NASA standards. While the rule-based SCORPEON model achieved an $R^2$ score of only 0.03, BERTground reached an $R^2$ of 0.99, indicating it captures nearly all variance in the background signal.
The model is particularly effective in "Extreme Background" scenarios (≥ 1000 photons/sec), where it achieved a Root Mean Squared Error (RMSE) of 25.02, compared to SCORPEON’s 36.62. This performance demonstrates that BERTground can handle the non-linear spikes that typically baffle simpler models.
Conclusion and Future Directions
By treating tabular data as a structured "language" of physical features, BERTground provides a robust solution for noise subtraction in NICER data. Future work will focus on incorporating temporal inductive bias—using LSTMs or temporal Transformers—to account for the "memory" of space weather events. Additionally, efforts are underway to integrate BERTground into HEASoft, the standard software suite for astrophysicists, making this tool available for global scientific use.
Summary updated