Accounting for noise and singularities in Bayesian calibration methods for global 21-cm cosmology experiments

C. Kirkham, Will Handley, Jiacong Zhu, Kaan Artuc, I. L. V. Roque, S. A. K. Leeney, H. T. J. Bevins, Dominic Anstey, Eloy de Lera AcedoMonthly 2024

TL;DR

This paper introduces three novel Bayesian calibration methods for 21-cm cosmology experiments, specifically addressing issues of instrumental noise and mathematical singularities that plague existing techniques. These new methods significantly improve calibration accuracy, particularly on real-world lab data, by more robustly handling noisy calibrator sources and avoiding problematic mathematical behaviors.

What Problem Does This Paper Address?

Global 21-cm cosmology experiments aim to detect faint signals from the early universe, but these signals are often buried under much stronger galactic foregrounds. To accurately extract the 21-cm signal, instruments must be meticulously calibrated to avoid instrumental systematics that can mimic or mask the cosmological signal. A key challenge in this calibration process lies in accurately characterizing the response of the low-noise amplifier (LNA) within the instrument. Existing methods, like the "conjugate priors" approach, struggle when dealing with calibrator sources that have very high reflection coefficients (leading to mathematical singularities) or significant internal noise. These issues can lead to biased calibration solutions, particularly affecting the estimation of the noise source temperature, which in turn impacts the accuracy of the final 21-cm signal measurement.

Methods

The authors propose three new Bayesian calibration methods, building upon the "noise wave parameter" (NWP) formalization:

  1. Γ-Weighted Conjugate Priors: This method modifies the existing conjugate priors approach by weighting each calibrator's contribution based on its reflection coefficient (Γ). This physically motivated weighting downweights problematic sources (like shorted loads with Γ close to 1), mitigating singularities and normalizing the noise contribution.
  2. Marginalised Polynomial Method: Instead of assuming fixed polynomial orders for the NWPs, this method treats the polynomial orders themselves as parameters to be sampled. This allows for a more flexible fitting process and avoids the computational burden and potential local minima issues of previous order-selection methods. It also analytically marginalizes over polynomial coefficients, speeding up computation.
  3. Marginalised Polynomial Method with Noise Model: This builds on the second method by incorporating a physically motivated noise model into the covariance matrix. This allows for a more accurate representation of the noise characteristics of the calibrators, further improving the calibration accuracy.

Datasets

The methods were tested using:

  • Simulated Mock Data: Five sets of simulated data were generated, mimicking the REACH receiver design. These datasets varied in complexity, including flat and power-law spectra, and considered different cable temperature scenarios.
  • Lab Data: A real-world dataset from a simplified version of the REACH system was used. This dataset included seven sources and was used to validate the performance of the calibration methods on actual experimental data.

Key Insights and Ideas

  • Singularities are a major problem: Calibrator sources with reflection coefficients close to unity (e.g., shorted loads) introduce mathematical singularities that severely bias existing calibration methods.
  • Noise matters more than you think: The goal of calibration isn't just to achieve the lowest residual noise (RMSE), but to have the RMSE be as close as possible to the expected noise. A significantly lower RMSE suggests the noise parameters have been incorrectly fitted.
  • Physically motivated weighting helps: Downweighting noisy or problematic calibrators based on their physical properties (like reflection coefficient) is a robust way to handle singularities and improve calibration stability.
  • Flexibility in model order is crucial: Treating polynomial orders as parameters to be sampled, rather than fixed, allows for a more accurate representation of the underlying physics and avoids overfitting.
  • Noise modeling is key: Incorporating a physically motivated noise model into the calibration likelihood significantly improves accuracy, especially when dealing with complex noise sources.

Key Results

  • The three proposed methods (Γ-weighted conjugate priors, marginalised polynomial, and marginalised polynomial with noise model) significantly outperform the existing conjugate priors method on the lab dataset.
  • On the lab dataset, the new methods achieved calibration accuracies within 5% of the noise floor.
  • The conjugate priors method was heavily biased by the noisy shorted load calibrator in the lab data, leading to incorrect noise wave parameter fits and an underestimated noise source temperature (TNS).
  • The new methods recovered a more realistic TNS value for the lab dataset (around 1100 K), closer to the expected value derived from the noise diode's ENR and ambient temperature.
  • For simulated data, all methods performed comparably, but the new methods showed a clear advantage on the more challenging lab data.

Limitations and Future Work

  • The authors note that the assumption of noiseless S11 measurements in the noise model could be improved.
  • Further investigation is needed into the unphysical negative values of Tunc observed in some mock datasets, which might be due to degeneracies in the NWP solutions that the new methods break.
  • The paper suggests that the open and short cables might not be strictly necessary for calibration with the Γ-weighted method, which warrants further exploration.
  • Achieving science-ready noise levels on the lab data would require longer integration times.

Summary updated

Abstract

ABSTRACT Due to the large dynamic range involved with separating the cosmological 21-cm signal from the Cosmic Dawn from galactic foregrounds, a well-calibrated instrument is essential to avoid biases from instrumental systematics. In this paper, we present three new methods for calibrating a global 21-cm cosmology experiment using the noise wave parameter formalization to characterize a low-noise amplifier including a careful consideration of how calibrator temperature noise and singularities will bias the result. The first method presented in this paper builds upon the existing conjugate priors method by weighting the calibrators by a physically motivated factor, thereby avoiding singularities and normalizing the noise. The second method fits polynomials to the noise wave parameters by marginalizing over the polynomial coefficients and sampling the polynomial orders as parameters. The third method introduces a physically motivated noise model to the marginalized polynomial method. Running these methods on a suite of simulated data sets based on the Radio Experiment for the Analysis of Cosmic Hydrogen receiver design and a lab data set, we found that our methods produced a calibration solution which is equally as or more accurate than the existing conjugate priors method when compared with an estimate of the calibrator’s noise. We find in the case of the measured lab data set the conjugate priors method is biased heavily by the large noise on the shorted load calibrator, resulting in incorrect noise wave parameter fits. This is mitigated by the methods introduced in this paper which calibrate the validation source spectra to within 5 per cent of the noise floor at best.
BibTeX
@inproceedings{kirkham2024accounting,
  author = {C. Kirkham and Will Handley and Jiacong Zhu and Kaan Artuc and I. L. V. Roque and S. A. K. Leeney and H. T. J. Bevins and Dominic Anstey and Eloy de Lera Acedo},
  title = {{Accounting for noise and singularities in Bayesian calibration methods for global 21-cm cosmology experiments}},
  booktitle = {Monthly},
  year = {2024},
}
Figure