Bits Missing: Finding Exotic Pulsars Using bfloat16 on NVIDIA GPUs

Jack White, Karel Adámek, Jayanta Roy, Sofia Dimoudi, S. M. Ransom, Wesley Armour

TL;DR

Researchers have successfully accelerated the search for exotic binary pulsars by 1.6x by switching from standard 32-bit floating-point math to the 16-bit bfloat16 format on NVIDIA GPUs. Despite the "missing bits," the system maintains high sensitivity, recovering signals with less than a 3% impact on the signal-to-noise ratio—a crucial trade-off for the real-time data demands of next-generation radio telescopes.

What Problem Does This Paper Address?

Searching for binary pulsars—pulsars in orbit with another object—is a computational nightmare. As these stars orbit, their signals are Doppler-shifted, causing the pulse period to "drift" over time. To find them, astronomers use the Fourier Domain Acceleration Search (FDAS), which involves convolving the signal with thousands of templates representing different possible accelerations.

The challenge is the sheer volume of data. Next-generation telescopes like the Square Kilometre Array (SKA) are expected to produce data at rates of 800 GB/s. Storing this for later analysis is impossible; we must process it in real-time. Current pipelines using standard IEEE-754 single-precision (FP32) math are often too slow or hardware-intensive to keep up with this "data deluge."

Methods

The authors modified the AstroAccelerate software package to utilize bfloat16 (Brain Floating Point) for the most demanding part of the FDAS pipeline: the Fourier domain convolutions.

Standard 16-bit "half-precision" (FP16) is often unsuitable for scientific computing because its narrow dynamic range leads to frequent overflows during Fast Fourier Transforms (FFTs). However, bfloat16 is a "truncated" version of FP32; it keeps the same 8-bit exponent (maintaining the same dynamic range) but reduces the mantissa (precision) from 23 bits to 7 bits.

The team implemented a mixed-precision pipeline where:

  1. The initial "Real FFT" is kept in FP32 to preserve signal integrity.
  2. The heavy lifting—complex multiplications and Inverse FFTs—is performed in bfloat16.
  3. The results are "quick-cast" back to FP32 by simply appending 16 zeros to the bitstring, avoiding expensive GPU kernel overhead.

Datasets

The study used two primary data sources:

  • Synthetic Data: Approximately 20,000 pulsar filterbank files generated using the SIGPROC software. These simulated a wide variety of physical parameters (period, width, acceleration) to test the limits of the numerical precision.
  • Real-World Data: Observations from the Giant Metrewave Radio Telescope (GMRT) of PSR J1544+4937, a "black widow" pulsar (a millisecond pulsar in a very tight, 2.8-hour orbit).

Key Insights and Ideas

  • Dynamic Range Trumps Precision: In pulsar searching, the ability to represent very large and very small numbers (dynamic range) is more important than the number of decimal places (precision). This is why bfloat16 succeeded where standard FP16 would likely fail due to overflows.
  • The "Noise Canopy": The authors found that "extreme" errors in detection (bin drifts) only occurred when the signal was already buried in the noise. If a signal is strong enough to be detected, the reduced precision of bfloat16 almost never moves it to the wrong search bin.
  • Zero-Cost Casting: Because bfloat16 is essentially the upper half of an FP32 number, converting between them is a trivial bit-shifting operation. This allows the pipeline to drop into lower precision for the "math-heavy" parts and pop back out without a performance penalty.

Key Results

  • Speedup: The bfloat16 implementation achieved a 1.6x speedup compared to the standard FP32 version.
  • Sensitivity: All bfloat16 peaks were within 3% of the predicted signal-to-noise ratio (SNR) of the FP32 "gold standard."
  • Positional Accuracy: For "bright" signals, 97.53% of detections stayed in the exact same frequency/acceleration bin. The remaining 2.47% only drifted by a single adjacent bin.
  • Real-World Validation: When tested on GMRT data, the bfloat16 pipeline successfully detected the black widow pulsar with values nearly identical to the high-precision results and the industry-standard PRESTO software.

Limitations and Future Work

While this paper focuses on FDAS (constant acceleration), many binary systems have changing acceleration, requiring a Jerk Search. The authors note that the computational demands of Jerk Searches are even higher, making the 1.6x speedup from bfloat16 even more attractive for future implementations.

Additionally, while the convolution is faster, the initial FFT remains in FP32. Future work could investigate if even earlier stages of the pipeline can be moved to reduced precision without losing the "exotic" pulsars that are the primary targets of these surveys.

Summary updated

Abstract

Abstract The Fourier domain acceleration search (FDAS) is an effective technique for detecting faint binary pulsars in large radio astronomy data sets. This paper quantifies the sensitivity impact of reducing numerical precision in the graphics processing unit (GPU)-accelerated FDAS pipeline of the AstroAccelerate (AA) software package. The prior implementation used IEEE-754 single-precision in the entire binary pulsar detection pipeline, spending a large fraction of the runtime computing GPU-accelerated fast Fourier transforms. AA has been modified to use bfloat16 (and IEEE-754 double-precision to provide a “gold standard” comparison) within the Fourier domain convolution section of the FDAS routine. Approximately 20,000 synthetic pulsar filterbank files representing binary pulsars were generated using SIGPROC with a range of physical parameters. They have been processed using bfloat16, single-precision, and double-precision convolutions. All bfloat16 peaks are within 3% of the predicted signal-to-noise ratio of their corresponding single-precision peaks. Of 14,971 “bright” single-precision fundamental peaks above a power of 44.982 (our experimentally measured highest noise value), 14,602 (97.53%) have a peak in the same acceleration and frequency bin in the bfloat16 output plane, while in the remaining 369 the nearest peak is located in the adjacent acceleration bin. There is no bin drift measured between the single- and double-precision results. The bfloat16 version of FDAS achieves a speedup of approximately 1.6× compared to single-precision. A comparison between AA and the PRESTO software package is presented using observations collected with the GMRT of PSR J1544+4937, a 2.16 ms black widow pulsar in a 2.8 hr compact orbit.
BibTeX
@article{white2023bits,
  author = {Jack White and Karel Adámek and Jayanta Roy and Sofia Dimoudi and S. M. Ransom and Wesley Armour},
  title = {{Bits Missing: Finding Exotic Pulsars Using bfloat16 on NVIDIA GPUs}},
  journal = {The Astrophysical Journal Supplement Series},
  year = {2023},
  doi = {10.3847/1538-4365/acb351},
}

Loading related content...

Figure