Efficient Supervision Strategy for Tomographic AO Systems on E-ELT

Nicolas Doucet, ‪Damien Gratadour‬, Hatem Ltaief, É. Gendron, Arnaud Sevin, Florian Ferreira, Fabrice Vidal, Ronald Kriemann, David E. KeyesProceedings of the Adaptive Optics for Extremely Large Telescopes 5 2017 · DOI

Abstract

A critical subsystem of the tomographic Adaptive Optics (AO) Real-Time Control (RTC) is the supervisor module. Its role is to feed the challenging real-time data pipeline with a new reconstructor matrix at a regular rate, computed from a statistical analysis of the measurements, to optimize the performance of the AO system. This process involves solving a system of linear equations defined by the covariance matrix of the wave front sensors' measurements, the size of which may be up to 90k x 90k for the E-ELT's first light instruments. The computational load for the solver of this dense symmetric matrix system is significant but can be efficiently handled using state-of-the-art energy-efficient manycore x86 or accelerator-based architectures. As part of the Green Flash project, we develop a supervisor module and demonstrate its portability by deploying it on various hardware systems. We describe implementations and trade-offs in performance and accuracy, including preliminary results on hierarchically low-rank approximation methods.

Introduction

Adaptive Optics (AO) systems are essential for ground-based telescopes to correct for atmospheric turbulence in real-time. As telescopes scale up to the 40-meter class of the Extremely Large Telescope (ELT), the computational burden for AO control systems increases dramatically. The "supervisor module" acts as the brain of this system, analyzing sensor measurements to periodically update the reconstructor matrix—the mathematical operator used to command deformable mirrors. For the ELT, this requires solving dense linear systems with dimensions up to 90,000 x 90,000 every few minutes, demanding a throughput of approximately 100 TFlops to track changing atmospheric conditions.

Methods and Implementation

The supervisor module implements a "Learn and Apply" pipeline designed for high-performance and portability across many-core architectures like GPUs and Intel Xeon-Phi (KNL).

The Supervisor Pipeline

The workflow is divided into two main stages:

  1. The Learn Pipeline: This stage identifies atmospheric properties such as altitude, strength, and turbulence scale. It employs the Levenberg-Marquardt (LM) algorithm to minimize the residual between measured and theoretical covariance matrices.
  2. The Apply Pipeline: Once atmospheric parameters are determined, the module computes the projector matrix ($R'$) by solving the dense linear system $R' \times Cmm = Ctm$.
The supervisor module diagram illustrates the data flow from raw measurements through the Learn and Apply stages to the final tomographic reconstructor.

Software Architecture

To ensure hardware portability, the module utilizes the Chameleon library and the StarPU runtime. This task-based approach decomposes mathematical operations into small, independent tasks that are dynamically scheduled across available processing units (CPUs or GPUs). This abstraction allows the same code to achieve high efficiency on diverse platforms without manual re-tuning.

Performance and Results

The system was evaluated using simulated data tailored to the E-ELT scale, featuring a 40m telescope with 9 wavefront sensors and approximately 86,688 measurements ($N \approx 86k$).

Throughput and Efficiency

On a DGX-1 system equipped with 8x NVIDIA P100 GPUs, the "Apply" pipeline solved a 100k x 100k matrix in 26 seconds, reaching roughly 50% of the hardware's peak performance. In comparison, a Haswell-based CPU system achieved ~1 TFlops/s, while the GPU-based system reached over 21 TFlops/s for the Apply phase.

Scaling Results

The "Learn" pipeline demonstrates excellent strong scaling. Efficiency remains above 90% when scaling from 2 to 8 GPUs for most configurations, indicating that the task-based scheduling effectively manages the computational load even as the number of accelerators increases.

Strong scaling results for the Chi2 and Hg components of the Learn pipeline, showing high efficiency across multiple GPU configurations.

Optimization and Future Directions

A key optimization explored is "XY filtering," where cross-axis correlations (lower in magnitude) are ignored to reduce memory footprint. Results indicate this significantly reduces data load without compromising the accuracy of the reconstructed atmospheric profile.

Furthermore, preliminary tests with Hierarchical (H-matrices) showed potential for up to 40% compression. However, challenges remain as the rank of matrix blocks tends to increase during factorization in the "Apply" phase. Future work will focus on deeper integration of low-rank methods to reduce real-time latency and further optimizing the "Learn" phase to under one minute to better track rapid atmospheric fluctuations.

Summary updated

Abstract

A critical subsystem of the tomographic AO RTC is the supervisor module. Its role is to feed the challenging real-time data pipeline with a new reconstructor matrix at a regular rate, computed from a statistical analysis of the measurements, to optimize the performance of the AO system. This process involves solving a system of linear equations defined by the covariance matrix of the wave front sensors' measurements, the size of which may be up to 90k90k for the E-ELT's first light instruments using tomographic AO modules. The computational load for the solver of this dense symmetric matrix system is quite significant at this scale but can be efficiently handled using state-of-the-art energy-efficient manycore x86 or accelerator-based architectures, such as KNLs or GPUs, respectively. As part of the Green Flash project, we develop a supervisor module and demonstrate its portability by deploying it on each aforementioned hardware system. Finally, we describe different implementations and their trade-offs in terms of performance and accuracy and show preliminary results on the possible impact of hierarchically low-rank approximation methods on the overall supervisor module.
BibTeX
@inproceedings{doucet2017efficient,
  author = {Nicolas Doucet and ‪Damien Gratadour‬ and Hatem Ltaief and É. Gendron and Arnaud Sevin and Florian Ferreira and Fabrice Vidal and Ronald Kriemann and David E. Keyes},
  title = {{Efficient Supervision Strategy for Tomographic AO Systems on E-ELT}},
  booktitle = {Proceedings of the Adaptive Optics for Extremely Large Telescopes 5},
  year = {2017},
  doi = {10.26698/ao4elt5.0099},
}
Figure