A Spectral Reconstruction Algorithm of Miniature Spectrometer Based on Sparse Optimization and Dictionary Learning

Shang Zhang, Yuhan Dong, Hongyan Fu, Shao-Lun Huang, Lin ZhangSensors 2018 · OpenReview

TL;DR

This paper presents a novel spectral reconstruction algorithm for miniature spectrometers that utilizes sparse optimization and dictionary learning. This approach enables the creation of smaller, cheaper spectrometers by using fewer filters while maintaining or improving spectral reconstruction accuracy, even for complex, non-directly sparse spectra.

What Problem Does This Paper Address?

Traditional spectrometers, while powerful, are often bulky and expensive due to their reliance on complex optical components like gratings or prisms. Miniaturizing spectrometers is a significant research goal to expand their applications in areas like environmental monitoring and medical diagnostics. One promising miniaturization strategy is using filter-based approaches, where a series of filters with distinct transmission functions are employed. However, fabricating ideal, narrow-band filters is impractical and costly. Real-world filters are often "broadband" with complex transmission profiles, making direct spectral readout impossible. This necessitates a "spectral reconstruction" step to recover the original spectrum from the filter outputs. Existing reconstruction methods, like least squares, struggle with the ill-conditioned nature of these systems and require a large number of filters for good resolution. This paper tackles the challenge of achieving accurate spectral reconstruction with a minimal number of filters in a filter-based miniature spectrometer.

Methods

The core of the proposed solution lies in combining two powerful signal processing techniques:

  • Sparse Optimization: This technique is particularly effective for solving underdetermined systems of linear equations, which is the case when the number of filters ($m$) is less than the spectral resolution ($n$). The paper formulates spectral reconstruction as an $l_1$-norm minimization problem. This approach is advantageous because it can recover sparse signals (signals with few non-zero components) even when the system is ill-posed and noisy. For directly sparse spectra (like those from narrowband filters), this method alone can yield good results.
  • Dictionary Learning: For spectra that are not inherently sparse (i.e., "non-directly sparse"), the paper introduces dictionary learning. This technique learns a "dictionary" (a set of basis vectors or "atoms") that can represent the complex spectra in a sparse manner. By transforming the spectra into this learned domain, their sparsity is enhanced, making them amenable to sparse optimization. The dictionary learning process involves iteratively optimizing the dictionary and the sparse representations of the training spectra. The paper specifically uses a non-negative dictionary learning approach, as natural spectra are non-negative.

The overall algorithm first uses dictionary learning to find a sparse representation for non-directly sparse spectra and then applies sparse optimization to reconstruct the spectrum.

Datasets

  • Training Data: The paper uses transmission functions of 210 filters as part of the training set. To augment this, they also create new training spectra by multiplying pairs of filter transmission functions, simulating overlapping filters. The total training set size is 3000 spectra. These filter transmission functions were sampled at 0.5 nm intervals. Additionally, they incorporated transmission spectra of various chemicals from a public database ([32]) and the spectrum of the light source used (halogen lamp or LED).
  • Evaluation Data: The paper evaluates its method on reconstructed spectra from a halogen lamp and a light-emitting diode (LED), both with and without additional filters. Ground truth spectra were obtained using a commercial scientific-grade spectrometer.

Key Insights and Ideas

  • Sparse Optimization is Key for Miniaturization: The paper highlights that sparse optimization is crucial for reducing the number of filters needed in a spectrometer. This directly translates to smaller size and lower cost.
  • Dictionary Learning Unlocks Sparsity for Complex Spectra: The most significant insight is that dictionary learning can transform non-directly sparse spectra into a sparse representation, making them solvable by sparse optimization. This overcomes a major limitation of previous sparse methods.
  • Learned Dictionary Outperforms Generic Kernels: The paper demonstrates that a dictionary learned specifically for spectral data significantly outperforms generic transform domains like Gaussian kernels, even when those kernels are carefully optimized. This suggests that domain-specific learning is vital for effective spectral reconstruction.
  • Non-negativity Constraint is Important: Incorporating non-negativity constraints into both dictionary learning and sparse optimization is essential for accurately representing natural optical spectra.
  • Fewer Filters Can Be Better (Sometimes): Surprisingly, the study found that in some cases, reducing the number of filters (from 210 to 20) could lead to improved reconstruction quality. This is attributed to the fact that the learned dictionary captures the essential spectral information, and too many filters might introduce more noise than useful data in this simplified setup.

Key Results

  • Directly Sparse Spectra: The algorithm achieved satisfactory reconstruction of narrowband spectra, accurately determining passbands and crests, with only minor perturbations due to measurement error.
  • Non-Directly Sparse Spectra:
    • Halogen Lamp: Reconstructed spectra showed consistent variation tendencies and similar shapes to the ground truth, with crests and troughs located at the same wavelengths, despite some discrepancies.
  • LED: Similar to the halogen lamp, the LED reconstruction showed minor deviations and consistent variation tendencies, suggesting LEDs are a viable, cost-effective light source.
  • Comparison with Gaussian Kernels: Dictionary learning achieved significantly lower relative reconstruction errors (5.92% vs. 21.09% for halogen lamp; 10.25% vs. 57.53% for LED) compared to optimized Gaussian kernels. Gaussian kernels also showed severe deviations from ground truth and were difficult to parameterize.
  • Effect of Filter Count: Reducing the number of filters from 210 to 20 sometimes led to improved reconstruction quality, suggesting that the learned dictionary effectively captures spectral information with fewer measurements.

Limitations and Future Work

  • Reconstruction Discrepancy: The authors acknowledge that the reconstruction results are still "slightly discrepant from the true spectra." This indicates room for algorithmic improvement.
  • Filter Property Requirements: The relationship between the number of filters and the required properties of their transmittance functions is not fully understood and needs further investigation.
  • Hardware Implementation: The paper suggests future work on improving the algorithm and designing compact hardware, specifically a miniature spectrometer with a proper filter array.
  • Imaging Spectrometer: The authors aim to implement an imaging spectrometer using a rotational filter array, which would enable spectral imaging capabilities.

Summary updated

Abstract

The miniaturization of spectrometer can broaden the application area of spectrometry, which has huge academic and industrial value. Among various miniaturization approaches, filter-based miniaturization is a promising implementation by utilizing broadband filters with distinct transmission functions. Mathematically, filter-based spectral reconstruction can be modeled as solving a system of linear equations. In this paper, we propose an algorithm of spectral reconstruction based on sparse optimization and dictionary learning. To verify the feasibility of the reconstruction algorithm, we design and implement a simple prototype of a filter-based miniature spectrometer. The experimental results demonstrate that sparse optimization is well applicable to spectral reconstruction whether the spectra are directly sparse or not. As for the non-directly sparse spectra, their sparsity can be enhanced by dictionary learning. In conclusion, the proposed approach has a bright application prospect in fabricating a practical miniature spectrometer.
BibTeX
@inproceedings{zhang2018spectral,
  author = {Shang Zhang and Yuhan Dong and Hongyan Fu and Shao-Lun Huang and Lin Zhang},
  title = {{A Spectral Reconstruction Algorithm of Miniature Spectrometer Based on Sparse Optimization and Dictionary Learning}},
  booktitle = {Sensors},
  year = {2018},
  url = {https://openreview.net/forum?id=moBjB654rPwG},
}

Loading related content...

Figure