Deep Optics for Optomechanical Control Policy Design

Justin FletcherWACV 2024

TL;DR

This paper introduces a "deep optics" approach to design optical systems for distributed aperture telescopes, which are cheaper and more scalable than traditional large telescopes. By jointly learning how to control the telescope's optics (using deformable mirrors) and how to process the resulting images, they achieve image recovery performance comparable to expensive monolithic telescopes, even in the presence of atmospheric disturbances.

What Problem Does This Paper Address?

Traditional large astronomical telescopes, while offering superior angular resolution, are prohibitively expensive to build and scale. Distributed aperture telescopes, which use multiple smaller mirrors spread out to achieve a large effective aperture, offer a more cost-effective path to higher resolution. However, these designs introduce significant challenges, particularly inter-aperture phase errors due to structural flexibility and atmospheric turbulence. Correcting these errors with conventional methods is complex and often requires precise knowledge of the system and the target. This paper addresses the problem of designing control policies for these distributed aperture telescopes that can compensate for these wavefront errors and achieve high-quality image recovery, even when the exact nature of the errors or the target object is not fully known beforehand.

Methods

The core innovation is a "deep optics" framework that treats the entire optical system and image recovery pipeline as a single, end-to-end trainable system.

  • Differentiable Optomechanical Simulation: They developed a proxy model of the optical system that is differentiable. This allows them to propagate error gradients from the image recovery stage back to the optical control parameters.
  • Joint Learning: The system jointly learns two key components:
  • Optomechanical Control Policy: This is a sequence of actuations for a deformable mirror (DM). The DM is used to dynamically shape the aperture, influencing how light interferes on the focal plane. Instead of correcting aberrations, the policy learns to exploit these dynamic changes to extract useful features.
  • Image Recovery Model: This is a neural network that takes the ensemble of images produced by the different DM actuations and reconstructs the original object.
  • Exploiting Aberrations: Rather than solely focusing on eliminating wavefront errors, the approach learns to use the dynamic changes in the aperture (driven by the DM actuations) to its advantage, effectively treating the DM as a "feature extractor."

Datasets

  • SPEED+ (Synthetic): A dataset of synthetic satellite images used for training and validation. This dataset was chosen to approximate an imaging task for satellites in low-Earth orbit.
  • iNaturalist (Mini Variant): A dataset of natural images (plants and animals) used for evaluating generalization performance, particularly in the presence of atmospheric turbulence. This dataset was chosen because the semantics of the objects do not directly influence the learning process, allowing for a broader test of generalization.

Key Insights and Ideas

  • Deformable Mirrors as Feature Extractors: The paper demonstrates that DMs, when actuated dynamically, can act as physical feature extractors, similar to how diffraction neural networks operate. By learning a sequence of actuations, the system can emphasize specific spatial frequencies relevant to the object.
  • Subverting the Correction Paradigm: Instead of viewing wavefront errors as something to be purely corrected, the approach learns to leverage them. The dynamic nature of the DM allows for adaptive aperture shaping that can enhance image recovery for specific object domains.
  • End-to-End Learning for Optics and Imaging: The joint learning of the optical control policy and the image recovery model is crucial. This allows the system to optimize both components simultaneously, leading to better overall performance than optimizing them separately.
  • Generalization with Model Capacity: Larger neural network models for image recovery demonstrate better generalization and improved performance, especially in the presence of atmospheric turbulence.

Key Results

  • Recovery Parity: The proposed approach achieves "recovery parity," meaning its performance matches that of a perfectly phased, filled aperture telescope of the same diameter. This is a significant achievement for distributed aperture systems.
  • Improved Metrics: The system shows significant improvements in MSE, SSIM, and PSNR compared to baseline methods, particularly when compared to a filled aperture telescope. For example, they report SSIM parity at around 130 training epochs on the SPEED+ dataset.
  • Atmospheric Turbulence Compensation: The learned policies and recovery models are shown to effectively compensate for atmospheric turbulence, a major challenge for ground-based telescopes.
  • Qualitative Improvements: Figure 6 and Figure 9 visually demonstrate the improved recovery of high spatial frequency features and overall object detail compared to traditional methods.

Limitations and Future Work

  • Single Object Domain Focus: While iNaturalist was used for generalization testing, the primary training was on specific domains (SPEED+). Future work could explore more diverse object domains.
  • Atmospheric Model Simplification: The paper uses a simplified atmospheric model. More complex and realistic atmospheric models could be investigated.
  • Future Work Directions:
    • Specialized task losses for objectives like direct exoplanet imaging and biosignature detection.
  • Adaptations for other optical information processing objectives.
  • Reformulating the problem as a visuomotor sequential decision-making task.

Summary updated

BibTeX
@inproceedings{fletcher2024deep,
  author = {Justin Fletcher},
  title = {{Deep Optics for Optomechanical Control Policy Design}},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision},
  year = {2024},
}
Figure