Optical design and characterization of an advanced computational imaging system

R. Hamilton Shepard, Christy Fernandez-Cull, Ramesh Raskar, Boxin Shi, Christopher Barsi, Hang ZhaoProceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE 2014 · DOI

TL;DR

This paper introduces two novel, low-cost optical testbeds for advanced computational imaging systems. These systems enable simultaneous and dynamic coding at both the pupil and image planes, paving the way for enhanced spatial, temporal, and depth super-resolution imaging.

What Problem Does This Paper Address?

Traditional imaging systems are limited by fixed optical designs and detector capabilities. Achieving higher resolution, faster frame rates, or greater depth of field often requires expensive custom optics or complex post-processing. The authors aim to overcome these limitations by developing flexible optical architectures that can dynamically encode information at both the pupil (where light enters the system) and image (where the sensor captures the light) planes. This allows for "computational imaging," where the optical design and image processing work together to achieve capabilities beyond what either could do alone. Specifically, they address the need for systems that can perform simultaneous object-plane coding (for oversampling and super-resolution) and pupil-plane coding (for wavefront engineering and enhanced imaging characteristics), while also enabling dynamic, time-synchronized encoding.

Methods

The researchers developed two distinct testbeds, both leveraging commercially available off-the-shelf (COTS) optical components to keep costs down.

  • Coded-Aperture Programmable Pixel Imaging (CAPPI) Testbed: This system focuses on image-plane coding for spatial super-resolution. It uses a Digital Micromirror Device (DMD) to dynamically create structured illumination patterns on the object. This patterned light is then imaged onto a specialized MIT Lincoln Laboratory focal plane array (Orthogonal Transfer CCD) that has non-uniform pixel sampling. The key innovation here is using the DMD for per-pixel oversampling and then employing compressive sensing algorithms for reconstruction, allowing for higher effective resolution than the native pixel count.
  • Simultaneous Dynamic Pupil Plane Coding with On-Chip Processing (PETEI) Testbed: This testbed tackles simultaneous pupil- and image-plane coding. For pupil-plane coding, it explores two options: a deformable mirror for wavefront coding (altering the shape of the light wavefront) or a DMD for pupil apodization (modulating the amplitude of light across the pupil). Image-plane coding is achieved through time-varying and synchronized modulation using a Digital Readout Integrated Circuit (DROIC) focal plane array with advanced on-chip processing. This testbed is designed to enable simultaneous spatial-spectral, spatial-temporal, and spatial-depth imaging.

Both testbeds were carefully designed to meet specific optical requirements, including pupil matching, illumination uniformity, and minimizing distortion, all while utilizing COTS components.

Datasets

The paper does not explicitly mention using pre-existing datasets for training or evaluation. Instead, the authors focus on the design and characterization of their novel hardware testbeds. They present simulated results and initial "first-light" imagery from their CAPPI and PETEI systems, which serve as benchmarks for their proposed methods.

Key Insights and Ideas

  • COTS Optics for Advanced Imaging: The paper demonstrates that sophisticated computational imaging capabilities, like simultaneous pupil and image plane coding, can be achieved using relatively low-cost, commercially available optical components, significantly reducing the barrier to entry for such research.
  • Dynamic Encoding is Key: The flexibility offered by dynamic elements like DMDs and deformable mirrors is crucial for enabling advanced imaging techniques that go beyond static optical setups.
  • Leveraging Focal Plane Array Capabilities: The authors effectively repurpose specialized MIT Lincoln Laboratory focal plane arrays (OTCCD and DROIC) with unique pixel architectures and on-chip processing to enhance the capabilities of their computational imaging systems.
  • Trade-offs in Optical Design: The paper highlights the critical trade-offs involved in optical design when using COTS components, particularly concerning aperture stop placement, aberration control, and field of view limitations.
  • Oversampling and Reconstruction: The CAPPI testbed shows how per-pixel oversampling, combined with compressive sensing, can lead to significant improvements in spatial resolution, effectively "super-resolving" the image.

Key Results

  • Two Functional Testbeds: The successful development and characterization of both the CAPPI and PETEI testbeds are presented.
  • CAPPI Oversampling: The CAPPI testbed demonstrated the ability to achieve up to 36x oversampling, with a practical limit of 6x oversampling due to lens MTF, suggesting potential for higher resolution with improved optics.
  • PETEI Flexibility: The PETEI testbed is shown to be capable of accommodating both deformable mirrors for wavefront coding and DMDs for amplitude coding in the pupil plane, offering a versatile platform for pupil-plane manipulation.
  • Simulation of Benefits: Simulations for the CAPPI testbed show significant improvements in high-resolution target recovery with increasing numbers of masks used for reconstruction.

Limitations and Future Work

  • Lens MTF Limitation: The current CAPPI testbed's resolution is limited by the Modulation Transfer Function (MTF) of the projection and imaging lenses. The authors suggest that replacing these with higher-resolution lenses could enable higher oversampling ratios.
  • Characterization of SLMs: While the PETEI testbed is developed, complete characterization of the SLMs (deformable mirror and DMD) is ongoing.
  • First-Light Imagery and Validation: The paper mentions that further characterization and "first-light" data with the testbeds are still in progress and will be discussed in subsequent publications. This implies that the full performance potential and validation of the systems are yet to be fully demonstrated.
  • Future Exploration: The authors intend to use these testbeds to explore novel sensing modalities for spatial, temporal, and depth super-resolution, as well as passive 3D ranging and increased depth of focus.

Summary updated

Abstract

We describe an advanced computational imaging system with an optical architecture that enables simultaneous and dynamic pupil-plane and image-plane coding accommodating several task-specific applications. We assess the optical requirement trades associated with custom and commercial-off-the-shelf (COTS) optics and converge on the development of two low-cost and robust COTS testbeds. The first is a coded-aperture programmable pixel imager employing a digital micromirror device (DMD) for image plane per-pixel oversampling and spatial super-resolution experiments. The second is a simultaneous pupil-encoded and time-encoded imager employing a DMD for pupil apodization or a deformable mirror for wavefront coding experiments. These two testbeds are built to leverage two MIT Lincoln Laboratory focal plane arrays – an orthogonal transfer CCD with non-uniform pixel sampling and on-chip dithering and a digital readout integrated circuit (DROIC) with advanced on-chip per-pixel processing capabilities. This paper discusses the derivation of optical component requirements, optical design metrics, and performance analyses for the two testbeds built.
BibTeX
@inproceedings{shepard2014optical,
  author = {R. Hamilton Shepard and Christy Fernandez-Cull and Ramesh Raskar and Boxin Shi and Christopher Barsi and Hang Zhao},
  title = {{Optical design and characterization of an advanced computational imaging system}},
  booktitle = {Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE},
  year = {2014},
  doi = {10.1117/12.2060725},
}
Figure