Quantization-aware Deep Optics for Diffractive Snapshot Hyperspectral Imaging

Lingen Li, Lizhi Wang, Weitao Song, Lei Zhang, Zhiwei Xiong, Hua Huang

Abstract

Traditional hyperspectral imaging (HSI) systems often struggle with the trade-off between temporal resolution and hardware complexity. Snapshot HSI using Diffractive Optical Elements (DOEs) offers a solution, but the gap between high-precision computational designs and discrete physical fabrication levels often degrades performance. This paper introduces Quantization-aware Deep Optics (QDO), a framework that integrates hardware manufacturing constraints directly into the end-to-end optimization loop. By employing quantization-aware training and an adaptive height-level mechanism, the proposed system achieves high-fidelity hyperspectral reconstruction even with low-precision (e.g., 4-level) fabrication, effectively bridging the "reality gap" in deep optics.

Introduction

Hyperspectral imaging (HSI) is a critical tool for capturing spatial and spectral information simultaneously, enabling applications in medical diagnostics, precision agriculture, and environmental monitoring. Unlike standard RGB cameras, HSI captures a 3D "data cube" across numerous narrow wavelength bands. However, conventional HSI systems typically require scanning or multiple exposures, which limits their use in dynamic scenes.

Snapshot HSI aims to capture the entire data cube in a single 2D sensor exposure. A common approach involves using a Diffractive Optical Element (DOE) to encode spectral information into a spatially multiplexed image, which is then decoded by a reconstruction algorithm. Modern "Deep Optics" techniques optimize both the DOE's physical profile and the reconstruction network end-to-end. However, a significant challenge remains: while optimization occurs in high-precision floating-point space, physical fabrication via lithography is restricted to a few discrete height levels. This "quantization mismatch" leads to substantial performance drops when theoretical designs are manufactured.

Methods

The researchers propose a Quantization-aware Deep Optics (QDO) framework to address the fabrication gap. The system consists of a diffractive imaging encoder, a computational decoder, and a quantization-aware DOE optimization module.

The overall pipeline of the proposed Quantization-aware Deep Optics (QDO) framework, showing the end-to-end optimization of the DOE and reconstruction network.

The framework introduces two primary innovations:

  1. Quantization-aware Training (QAT): Instead of post-hoc quantization, the model incorporates the discrete nature of the DOE during training. Using an alpha-blending technique, the system gradually transitions from a full-precision height map to a quantized one. This allows the reconstruction network (a Res-UNet architecture) to learn and compensate for the specific optical artifacts introduced by the discrete fabrication levels.
  2. Adaptive Quantization (QDO+A): Standard quantization typically assumes evenly spaced levels. However, lithography allows for custom heights for each step. QDO+A treats these physical heights as trainable parameters. By optimizing the specific height of each fabrication level alongside the DOE pattern, the system minimizes information loss and maximizes reconstruction accuracy.

Results

The QDO framework was evaluated using the ICVL hyperspectral dataset and validated with a physical prototype. In synthetic experiments with only 4 quantization levels, the QDO+A method achieved a Peak Signal-to-Noise Ratio (PSNR) of 35.32 dB, a significant improvement over the 31.68 dB achieved by standard Deep Optics approaches. Remarkably, this 4-level adaptive model performed nearly as well as the theoretical "Full" precision model (36.82 dB).

Qualitative comparison of reconstruction results and spectral curves for different methods (DO, QDO, QDO+A) against Ground Truth.

Qualitative results demonstrate that QDO+A produces reconstructions with fewer artifacts and more accurate spectral signatures compared to non-quantization-aware methods. Physical validation using a fabricated 4-level DOE confirmed these findings, with the prototype successfully reconstructing spectral curves that closely matched ground-truth measurements from a commercial spectrometer.

Conclusion

This work demonstrates that hardware limitations, such as discrete fabrication levels, do not have to be a bottleneck for Deep Optics. By treating these constraints as learnable parameters within a quantization-aware framework, researchers can design snapshot HSI systems that are both high-performing and physically realizable. Future work may extend this framework to other optical tasks, such as high-dynamic-range imaging and depth sensing, further narrowing the gap between computational design and physical manufacturing.

Summary updated

Abstract

Diffractive snapshot hyperspectral imaging based on the deep optics framework has been striving to capture the spectral images of dynamic scenes. However, existing deep optics frameworks all suffer from the mismatch between the optical hardware and the reconstruction algorithm due to the quantization operation in the diffractive optical element (DOE) fabrication, leading to the limited performance of hyperspectral imaging in practice. In this paper, we propose the quantization-aware deep optics for diffractive snapshot hyperspectral imaging. Our key observation is that common lithography techniques used in fabricating DOEs need to quantize the DOE height map to a few levels, and can freely set the height for each level. Therefore, we propose to integrate the quantization operation into the DOE height map optimization and design an adaptive mechanism to adjust the physical height of each quantization level. According to the optimization, we fabricate the quantized DOE directly and build a diffractive hyperspectral snapshot imaging system. Our method develops the deep optics framework to be more practical through the awareness of and adaptation to the quantization operation of the DOE physical structure, making the fabricated DOE and the reconstruction algorithm match each other systematically. Extensive synthetic simulation and real hardware experiments validate the superior performance of our method.
BibTeX
@inproceedings{li2022quantization,
  author = {Lingen Li and Lizhi Wang and Weitao Song and Lei Zhang and Zhiwei Xiong and Hua Huang},
  title = {{Quantization-aware Deep Optics for Diffractive Snapshot Hyperspectral Imaging}},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year = {2022},
  doi = {10.1109/cvpr52688.2022.01916},
}
Figure