Planck 2018 results. III. High Frequency Instrument data processing and frequency maps
N. Aghanim, Y. Akrami, M. Ashdown, J. Aumont, C. Baccigalupi, M. Ballardini, A. J. Banday, R. B. Barreiro, N. Bartolo, S. Basak, K. Benabed, J.-p. Bernard, M. Bersanelli, P. Bielewicz, J. R. Bond, J. Borrill, F. R. Bouchet, F. Boulanger, M. Bucher, C. Burigana, Victoria Calafut, J.-f. Cardoso, Julien Carron, A. Challinor, H.c. Chiang, L. P. L. Colombo, Céline Combet, F. Couchot, B. P. Crill, F. Cuttaia, P. de Bernardis, A. de Rosa, G. de Zotti, J.‐m. Delouis, Eleonora Di Valentino, J. M. Diego, M. Douspis, A. Ducout, X. Dupac, G. Efstathiou, F. Elsner, T. A. Enßlin, H.k. Eriksen, E. Falgarone, Y. Fantaye, F. Finelli⋆, M. Frailis, A. A. Fraisse, E. Franceschi, Katherine Freese, S. Galeotta, S. Galli, K. Ganga, R.t. Génova-Santos, M. Gerbino, T. Ghosh, J. González-Nuevo, K. M. Górski, S. Gratton, A. Gruppuso, J.e. Gudmundsson, Will Handley, F. K. Hansen, S. Henrot–Versillé, D. Herranz, E. Hivon, Zhiqi Huang, Andrew H. Jaffe, W. C. Jones, A. Karakci, E. Keihänen, R. Keskitalo, K. Kiiveri, J. Kim, T. S. Kisner, N. Krachmalnicoff, M. Kunz, H. Kurki‐Suonio, G. Lagache, J.‐m. Lamarre, A. Lasenby, M. Lattanzi, C. R. Lawrence, F. Levrier, M. Liguori, P. B. Lilje, V. Lindholm, M. López-Caniego, Yin-Zhe Ma, J. F. Macías–Pérez, G. Maggio, D. Maino, N. Mandolesi, A. Mangilli, P. G. Martin, E. Martínez-González, S. Matarrese, N. MauriarXiv2018 · DOI · arXiv
This paper presents the High Frequency Instrument (HFI) data processing procedures for the Planck 2018 release. Major improvements in mapmaking have been achieved since the previous 2015 release, enabling the first significant measurement of the reionization optical depth parameter using HFI data. The paper presents an extensive analysis of systematic effects, including the use of simulations to facilitate their removal and characterize residuals. Polarized data are significantly improved, and calibration based on the CMB dipole is now extremely accurate. The Solar dipole direction and amplitude have been determined with unprecedented precision, providing a major legacy for future CMB experiments.
Introduction: Addressing Systematic "Leakage"
While the Planck mission concluded observations in 2013, the 2015 data release left several "known problems" unresolved, particularly in polarization data at large angular scales. The primary challenge was systematic errors—instrumental artifacts that mimic or obscure the faint signal of the Cosmic Microwave Background (CMB).
Specifically, the 2015 maps suffered from "leakage," where the much stronger intensity signal (temperature) bled into the polarization signal due to tiny mismatches in detector calibration and bandpass responses. These errors were larger than the actual cosmological signal, preventing researchers from accurately determining the reionization optical depth ($\tau$)—a key parameter describing when the first stars turned on and ionized the universe's gas.
Methods: The HFI Processing Pipeline and SRoll
The core of the 2018 improvement is a new mapmaking solution called SRoll. Unlike previous pipelines that relied heavily on ground-based measurements of detector properties, SRoll is an "integrated scheme" that uses the redundancy of the flight data itself to solve for instrumental parameters.
The HFI data processing pipeline, illustrating the sequence of corrections applied to raw bolometer data to produce Time-Ordered Information (TOI) and final map products.
The transition from raw bolometer data to final map products involves multiple stages of correction, including ADC non-linearity correction, demodulation, and glitch removal. SRoll then processes the resulting Time-Ordered Information (TOI) to produce frequency maps.
Quantities and their sources within the SRoll algorithm. Many key parameters, such as gain ($g_{b,k}$) and polarization components ($I_p, Q_p, U_p$), are solved directly within the mapmaking process.
Extended Destriping: SRoll simultaneously solves for pointing offsets, empirical transfer functions, and calibration mismatches between detectors.
Bandpass Mismatch Extraction: The algorithm extracts coefficients for foregrounds (like dust and CO gas) during the mapmaking process, ensuring that detectors with slightly different frequency responses are normalized to a common average.
End-to-End (E2E) Simulations: The team developed the FFP10 (Full Focal Plane) simulation suite. These model every known systematic effect to characterize residuals and validate that the cleaning process doesn't introduce new biases.
Key Insights and Results
The "aha moment" for the team was realizing that ground-based measurements were not accurate enough for the required precision. By letting the sky data "speak for itself" through the SRoll solver, intensity-to-polarization leakage was reduced by an order of magnitude.
Major results include:
Calibration Accuracy: Absolute calibration is now accurate to the $10^{-4}$ level for the 100–353 GHz channels.
Solar Dipole Precision: The direction is determined to within one arc minute, and the amplitude uncertainty is smaller than 0.35 $\mu$K. This provides a "standard candle" for future CMB experiments.
Reionization Measurement: These improvements directly enabled the HFI-based measurement of the reionization optical depth, $\tau$, crucial for understanding the early universe's timeline.
Limitations and Future Work
Despite these advancements, some "zebra" band striping remains in the 353 GHz maps due to incomplete correction of very long time-constant transfer functions. Additionally, residual calibration biases still exist at the $10^{-3}$ level between the first and second acoustic peaks of the CMB.
The authors note that while this is the final Planck collaboration release, the methods developed here—especially the E2E simulation framework and the SRoll algorithm—provide a roadmap for the next generation of CMB observatories to reach even higher sensitivity.
Summary updated
Abstract
This paper presents the High Frequency Instrument (HFI) data processing procedures for the Planck 2018 release. Major improvements in mapmaking have been achieved since the previous 2015 release. They enabled the first significant measurement of the reionization optical depth parameter using HFI data. This paper presents an extensive analysis of systematic effects, including the use of simulations to facilitate their removal and characterize the residuals. The polarized data, which presented a number of known problems in the 2015 Planck release, are very significantly improved. Calibration, based on the CMB dipole, is now extremely accurate and in the frequency range 100 to 353 GHz reduces intensity-to-polarization leakage caused by calibration mismatch. The Solar dipole direction has been determined in the three lowest HFI frequency channels to within one arc minute, and its amplitude has an absolute uncertainty smaller than $0.35\mu$K, an accuracy of order $10^{-4}$. This is a major legacy from the HFI for future CMB experiments. The removal of bandpass leakage has been improved by extracting the bandpass-mismatch coefficients for each detector as part of the mapmaking process; these values in turn improve the intensity maps. This is a major change in the philosophy of "frequency maps", which are now computed from single detector data, all adjusted to the same average bandpass response for the main foregrounds. Simulations reproduce very well the relative gain calibration of detectors, as well as drifts within a frequency induced by the residuals of the main systematic effect. Using these simulations, we measure and correct the small frequency calibration bias induced by this systematic effect at the $10^{-4}$ level. There is no detectable sign of a residual calibration bias between the first and second acoustic peaks in the CMB channels, at the $10^{-3}$ level.
BibTeX
@misc{aghanim2018planck,
author = {N. Aghanim and Y. Akrami and M. Ashdown and J. Aumont and C. Baccigalupi and M. Ballardini and A. J. Banday and R. B. Barreiro and N. Bartolo and S. Basak and K. Benabed and J.-P. Bernard and M. Bersanelli and P. Bielewicz and J. R. Bond and J. Borrill and F. R. Bouchet and F. Boulanger and M. Bucher and C. Burigana and Victoria Calafut and J.-F. Cardoso and Julien Carron and A. Challinor and H.C. Chiang and L. P. L. Colombo and Céline Combet and F. Couchot and B. P. Crill and F. Cuttaia and P. de Bernardis and A. de Rosa and G. de Zotti and G. de Zotti and J.‐M. Delouis and Eleonora Di Valentino and J. M. Diego and J. M. Diego and M. Douspis and A. Ducout and X. Dupac and G. Efstathiou and F. Elsner and T. A. Enßlin and H.K. Eriksen and E. Falgarone and Y. Fantaye and F. Finelli⋆ and M. Frailis and A. A. Fraisse and E. Franceschi and Katherine Freese and S. Galeotta and S. Galli and K. Ganga and R.T. Génova-Santos and M. Gerbino and T. Ghosh and J. González-Nuevo and K. M. Górski and S. Gratton and A. Gruppuso and J.E. Gudmundsson and Will Handley and F. K. Hansen and S. Henrot–Versillé and D. Herranz and E. Hivon and Zhiqi Huang and Andrew H. Jaffe and W. C. Jones and A. Karakci and E. Keihänen and R. Keskitalo and K. Kiiveri and J. Kim and T. S. Kisner and N. Krachmalnicoff and M. Kunz and H. Kurki‐Suonio and G. Lagache and J.‐M. Lamarre and A. Lasenby and M. Lattanzi and C. R. Lawrence and F. Levrier and M. Liguori and P. B. Lilje and V. Lindholm and M. López-Caniego and Yin-Zhe Ma and J. F. Macías–Pérez and G. Maggio and D. Maino and N. Mandolesi and A. Mangilli and P. G. Martin and E. Martínez-González and S. Matarrese and N. Mauri},
title = {{Planck 2018 results. III. High Frequency Instrument data processing and frequency maps}},
year = {2018},
eprint = {1807.06207},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/1807.06207},
}