KRISM - Krylov Subspace-based Optical Computing of Hyperspectral Images

Vishwanath Saragadam, Aswin C. Sankaranarayanan

Abstract

KRISM (Krylov Subspace-based Imaging and SpectroMetry) is a novel hyperspectral imaging architecture that leverages optical computing to adaptively find a low-rank approximation of a scene. By iterating between spatial and spectral coding, KRISM captures high-resolution hyperspectral data with significantly fewer measurements than traditional scanning methods. This approach effectively computes the scene's principal components directly in the optical domain, achieving up to 20x speedup while maintaining high fidelity.

Introduction

Hyperspectral imaging (HSI) provides a rich three-dimensional data cube (2D spatial + 1D spectral), but acquiring this data at high resolution is traditionally slow and light-inefficient. Conventional methods like tunable filters or pushbroom cameras scan one dimension at a time, often resulting in low signal-to-noise ratios or long exposure times. However, natural scenes often exhibit low-rank properties, meaning their spectral information is redundant across spatial pixels. KRISM addresses this by exploiting this redundancy during acquisition, measuring only the most significant dimensions of the scene's data cube.

The KRISM Architecture

The core innovation of KRISM is the physical implementation of the Lanczos bidiagonalization—a Krylov subspace method—directly in hardware. By treating the hyperspectral image as a matrix $X$, the system alternates between two optical operators:

  1. Spectrally-coded imager ($\mathcal{I}$): Produces a spatial image from a given spectral code, corresponding to the matrix-vector product $Xx$.
  2. Spatially-coded spectrometer ($\mathcal{S}$): Produces a spectral measurement from a given spatial code, corresponding to $X^\top \mathbf{x}$.
The KRISM architecture utilizes two primary optical operators to adaptively sense the dominant singular vectors of a scene.

The hardware setup uses a single Liquid Crystal on Silicon (LCoS) Spatial Light Modulator (SLM) and polarization-switching liquid crystal rotators to toggle between these modes. A key feature is the use of optimized coded apertures in the pupil plane, which allows for high spatial and spectral resolution without the light-loss constraints of traditional slits.

Experimental Results

Validation was performed using both simulations (KAIST, Harvard, ICVL, NASA AVIRIS, SpecTIR) and a physical prototype. KRISM demonstrated a 20x speedup over Nyquist sampling for rank-4 approximations. In simulations, a rank-10 approximation consistently yielded PSNR values above 40dB, indicating near-perfect reconstruction across various natural and aerial datasets.

Quantitative performance of KRISM across multiple datasets, showing PSNR improvement as the rank of the approximation increases.

Real-world tests confirmed high spectral accuracy, with narrow emission peaks of a CFL lamp matching professional spectrometer readings with a low Spectral Angular Mapper (SAM) error of 14.7°. Spatial resolution also saw significant improvements, jumping from 20 to 90 line pairs/mm through deconvolution of the coded aperture.

Discussion and Future Work

KRISM's adaptive sensing approach is more efficient than random-projection-based compressive sensing. By directly measuring singular vectors, it eliminates the need for computationally expensive post-processing. However, its efficiency depends on the scene's low-rank nature; highly complex scenes with many distinct materials may reduce its performance advantage. Future work aims to extend this to video-rate imaging and more complex "Sparse + Low-Rank" models to capture small, spectrally unique objects.

Summary updated

Abstract

We present an adaptive imaging technique that optically computes a low-rank approximation of a scene's hyperspectral image, conceptualized as a matrix. Central to the proposed technique is the optical implementation of two measurement operators: a spectrally-coded imager and a spatially-coded spectrometer. By iterating between the two operators, we show that the top singular vectors and singular values of a hyperspectral image can be adaptively and optically computed with only a few iterations. We present an optical design that uses pupil plane coding for implementing the two operations and show several compelling results using a lab prototype to demonstrate the effectiveness of the proposed hyperspectral imager.
BibTeX
@article{saragadam2019krism,
  author = {Vishwanath Saragadam and Aswin C. Sankaranarayanan},
  title = {{KRISM - Krylov Subspace-based Optical Computing of Hyperspectral Images}},
  journal = {ACM Trans. Graphics},
  year = {2019},
  url = {https://openreview.net/forum?id=CCqMjT-Cd-},
}
Figure