KRISM - Krylov Subspace-based Optical Computing of Hyperspectral Images

Vishwanath Saragadam, Aswin C. Sankaranarayanan

TL;DR

KRISM (Krylov Subspace-based Imaging and SpectroMetry) is a novel hyperspectral imaging architecture that uses optical computing to adaptively find a low-rank approximation of a scene. By iterating between spatial and spectral coding, it captures high-resolution hyperspectral data with up to 20x fewer measurements than traditional scanning methods, effectively "computing" the scene's principal components directly in the optical domain.

What Problem Does This Paper Address?

Hyperspectral images (HSIs) capture a scene across hundreds of narrow spectral bands, providing a "data cube" (2D space + 1D spectrum). Traditional capture methods face a fundamental trade-off: you can either scan one spectral band at a time (tunable filters) or one spatial row at a time (pushbroom cameras). At high resolutions, the amount of light reaching each individual "voxel" (spatial-spectral pixel) is tiny, leading to noisy images or requiring painfully long exposure times.

However, most natural scenes are "low-rank"—they are composed of just a few materials with distinct spectral signatures. The authors tackle the challenge of how to exploit this redundancy during the acquisition process. Instead of measuring every voxel and then compressing the data, can we optically measure only the most important "dimensions" of the scene?

Methods

The core innovation of KRISM is the optical implementation of the Krylov subspace method (specifically, Lanczos bidiagonalization). By treating the HSI as a large matrix $X$ (where rows are spatial pixels and columns are spectral bands), the authors implement two key optical operators:

  1. Spectrally-coded imager ($\mathcal{I}$): Given a spectral code (vector), it produces a spatial image. This corresponds to the matrix-vector product $Xx$.
  2. Spatially-coded spectrometer ($\mathcal{S}$): Given a spatial code, it produces a spectral measurement. This corresponds to the product $X^\top \mathbf{x}$.

By alternating between these two operators—using the output of one to inform the code for the next—the system adaptively "homes in" on the scene's dominant singular vectors.

To make this work in a single hardware setup, the authors designed a sophisticated optical path using a single Liquid Crystal on Silicon (LCoS) Spatial Light Modulator (SLM) and polarization-switching liquid crystal rotators. Crucially, they use optimized coded apertures in the pupil plane. This allows them to bypass the traditional "slit" requirement of spectrometers, enabling high spatial and spectral resolution simultaneously.

Datasets

The authors validated KRISM using both simulations and a physical lab prototype:

  • Simulated Datasets: KAIST, Harvard, ICVL, NASA AVIRIS, and SpecTIR. These provided a range of spatial resolutions and spectral depths (up to 260 bands).
  • Real-World Lab Scenes: The authors captured diverse objects including colorful dice, "Crayons," "Feathers," a "Color Checker" chart, and a toy figurine ("Ace") illuminated by a peaky Compact Fluorescent Lamp (CFL) to test spectral resolving power.

Key Insights and Ideas

  • Adaptive Sensing vs. Random Projections: Unlike standard Compressive Sensing (CS) which uses fixed, random masks, KRISM is adaptive. The measurement operator changes at each iteration based on previous data, making it far more efficient at capturing the scene's energy.
  • Optical Matrix-Vector Multiplication: The paper elegantly maps abstract linear algebra (SVD/Krylov methods) onto physical optical components. The SLM acts as the "vector" and the scene itself acts as the "matrix."
  • Pupil Plane Coding: By placing the coded aperture in the pupil plane rather than the image plane, the authors create a system where the spectral blur and spatial blur are both invertible. This solves the "slit vs. open aperture" dilemma that usually plagues hyperspectral design.
  • Zero Post-Processing: Because the system directly measures the singular vectors, the "reconstruction" is essentially just a summation of rank-1 approximations. This is orders of magnitude faster than the complex optimization algorithms required by other compressive imagers.

Key Results

  • Compression & Speed: KRISM achieved a 20x speedup over traditional Nyquist sampling for a rank-4 approximation. For example, a 49-minute Hadamard scan was reduced to under 2 minutes with KRISM.
  • High Fidelity: In simulations across multiple datasets, a rank-10 approximation yielded a PSNR higher than 40dB, indicating near-perfect reconstruction for most natural scenes.
  • Spectral Accuracy: Real-world measurements of peaky CFL spectra matched professional spectrometer readings with a Spectral Angular Mapper (SAM) error of only 14.7°, accurately capturing narrow emission peaks.
  • Spatial Resolution: Deconvolution of the coded aperture results showed a jump in spatial resolution from 20 line pairs/mm to 90 line pairs/mm.

Limitations and Future Work

  • Low-Rank Dependency: The method's efficiency relies entirely on the scene being low-rank. If a scene has high spectral complexity (e.g., hundreds of different materials), KRISM loses its advantage.
  • Photon Noise: As the algorithm progresses to higher-order singular vectors, it progressively blocks more light to find subtle features. This makes higher-rank measurements more susceptible to photon noise.
  • Hardware Complexity: The current prototype requires precise calibration of the SLM, camera, and polarization rotators.
  • Future Directions: The authors suggest extending the work to video-rate hyperspectral imaging and exploring "Sparse + Low-Rank" models to capture small, spectrally unique objects that might be missed by a pure low-rank approximation.

Summary updated

Abstract

We present an adaptive imaging technique that optically computes a low-rank approximation of a scene’s hyperspectral image, conceptualized as a matrix. Central to the proposed technique is the optical implementation of two measurement operators: a spectrally coded imager and a spatially coded spectrometer. By iterating between the two operators, we show that the top singular vectors and singular values of a hyperspectral image can be adaptively and optically computed with only a few iterations. We present an optical design that uses pupil plane coding for implementing the two operations and show several compelling results using a lab prototype to demonstrate the effectiveness of the proposed hyperspectral imager.
BibTeX
@article{saragadam2019krism,
  author = {Vishwanath Saragadam and Aswin C. Sankaranarayanan},
  title = {{KRISM - Krylov Subspace-based Optical Computing of Hyperspectral Images}},
  journal = {ACM Trans. Graphics},
  year = {2019},
  url = {https://openreview.net/forum?id=CCqMjT-Cd-},
}

Loading related content...

Figure