Full-Stokes polarization multispectral images of various stereoscopic objects

Axin Fan, Tingfa Xu, Geer Teng, Xi Wang, Yuhan Zhang, Chang Xu, Xin Xu, Jianan LiScientific 2023 · DOI

This paper, published in Scientific Data, introduces a comprehensive new resource for the optics and computer vision communities: a high-quality database of Full-Stokes polarization multispectral images. By providing standardized data for 67 different stereoscopic objects, the authors aim to eliminate the "data bottleneck" that currently hinders the development of real-time polarization imaging technologies.

TL;DR

The authors have released a first-of-its-kind database containing Full-Stokes polarization multispectral images (FSPMI) for 67 diverse stereoscopic objects across 18 spectral bands. This open-access dataset provides the high-fidelity "ground truth" necessary to simulate, develop, and benchmark next-generation polarization cameras and reconstruction algorithms.

What Problem Does This Paper Address?

Polarization Multispectral Imaging (PMI) is a powerful tool that captures four-dimensional information (spatial, spectral, and polarization). It allows researchers to see "hidden" features of an object, such as surface texture, nitrogen content in leaves, or cancerous microstructures in tissue, that are invisible to standard cameras.

However, traditional PMI systems are slow because they require scanning through different wavelengths and polarization states. To build faster "snapshot" or "compressive" cameras, researchers need high-quality FSPMI data to run simulations and validate their designs. Until now, no large-scale, standardized database existed. Researchers had to build their own complex experimental setups just to get a few test images, making it nearly impossible to compare different imaging methods fairly.

Methods

The researchers built a sophisticated experimental "cage system" in a dark room to capture the data. The setup consists of:

  • Modulation Hardware: A rotating quarter-wave plate (QWP) and a linear polarizer (LP) to manipulate the polarization of light reflected from the object.
  • Spectral Hardware: 18 switchable bandpass filters covering the 520 nm to 690 nm range (at 10 nm intervals).
  • The Capture Process: For each object, they captured images at five different QWP angles across all 18 spectral bands. To ensure data quality, they averaged six images per state to reduce noise and subtracted "dark noise" (images taken with the light source off).
  • Mathematical Modeling: They used Mueller matrix calculus to derive the four Stokes parameters ($S_0, S_1, S_2, S_3$), which fully describe the polarization state of light, including circular polarization.

Datasets

The authors introduced the FSPMI Dataset, which is publicly available via Figshare.

  • Content: 67 stereoscopic plastic objects (mostly refrigerator magnets and 3D-printed items) with varying shapes and textures.
  • Scale: A total of 36,582 raw images, totaling 571 GB of data.
  • Format: Processed data is provided in .mat files (512 × 512 pixels), including Stokes parameters and derived metrics like the Degree of Polarization (DoP) and Angle of Linear Polarization (AoLP).

Key Insights and Ideas

  • The "Full-Stokes" Advantage: Unlike many existing datasets that only look at linear polarization ($S_1, S_2$), this dataset includes $S_3$ (circular polarization). This is critical for advanced applications in biophotonics and material science.
  • Standardization as a Catalyst: By providing a uniform "ground truth," the community can finally move away from "one-off" experiments and toward rigorous benchmarking of reconstruction algorithms.
  • Surface vs. Appearance: The results highlight that while the $S_0$ parameter shows the general appearance of an object, the polarization parameters (like DoLP) reveal intricate surface textures and geometries that are otherwise washed out.
  • Enabling Machine Learning: The diversity of 67 objects provides a sufficient foundation for training deep learning models to reconstruct full polarization information from limited measurements—a key step toward real-time PMI.

Key Results

To prove the dataset's utility, the authors used it to validate three existing compressive sensing methods (CFPMI, FDCSPI, and FPCMI).

  • Reliability: The reconstruction of the Stokes parameters across all 67 objects achieved high fidelity, with an average Peak Signal-to-Noise Ratio (PSNR) greater than 20 dB and a Structural Similarity Index (SSIM) greater than 0.6.
  • Consistency: The data showed that $S_0$ (intensity) is the easiest to reconstruct, while $S_1, S_2,$ and $S_3$ (polarization components) require more sophisticated algorithms, highlighting the dataset's value as a challenging benchmark.

Limitations and Future Work

  • Material Diversity: The current dataset is limited to plastic objects. Future iterations could include metals, liquids, or biological tissues, which interact with polarized light differently.
  • Spectral Range: The study is confined to the visible/near-infrared spectrum (520–690 nm). Expanding into the ultraviolet or deeper infrared would be beneficial for remote sensing.
  • Image Registration: The authors note that slight shifts can occur when switching filters, suggesting that users may need to apply image registration (alignment) for high-precision applications.
  • Static Scenes: The dataset features stationary objects; capturing dynamic, moving scenes in full-Stokes multispectral remains a significant future challenge for the field.

Summary updated

Abstract

Polarization multispectral imaging (PMI) has been applied widely with the ability of characterizing physicochemical properties of objects. However, traditional PMI relies on scanning each domain, which is time-consuming and occupies vast storage resources. Therefore, it is imperative to develop advanced PMI methods to facilitate real-time and cost-effective applications. In addition, PMI development is inseparable from preliminary simulations based on full-Stokes polarization multispectral images (FSPMI). Whereas, FSPMI measurements are always necessary due to the lack of relevant databases, which is extremely complex and severely limits PMI development. In this paper, we therefore publicize abundant FSPMI with 512 × 512 spatial pixels measured by an established system for 67 stereoscopic objects. In the system, a quarter-wave plate and a linear polarizer are rotated to modulate polarization information, while bandpass filters are switched to modulate spectral information. The required FSPMI are finally calculated from designed 5 polarization modulation and 18 spectral modulation. The publicly available FSPMI database may have the potential to greatly promote PMI development and application.
BibTeX
@inproceedings{fan2023full,
  author = {Axin Fan and Tingfa Xu and Geer Teng and Xi Wang and Yuhan Zhang and Chang Xu and Xin Xu and Jianan Li},
  title = {{Full-Stokes polarization multispectral images of various stereoscopic objects}},
  booktitle = {Scientific},
  year = {2023},
  doi = {10.1038/s41597-023-02184-1},
}
Figure