Optimizing the ordering of the Hadamard masks of ghost imaging suitable for the efficient face reconstruction using the max-projection method

Haipeng Zhang, Kang Du, Changzhe Zhao, Jie Tang, Shangyu Si, Wenhong Jia, Lian Xue, Zhong‐Liang LiScientific 2023 · DOI

TL;DR

Researchers have developed a "max-projection" method to reorder Hadamard masks for ghost imaging, significantly boosting reconstruction quality at ultralow sampling rates. By using a tiny training set of just 20 images to prioritize the most informative patterns, the system can reconstruct high-fidelity face images at a 5% sampling rate in mere milliseconds.

What Problem Does This Paper Address?

Ghost imaging (GI)—also known as single-pixel imaging—is a clever technique that captures images using a series of structured light patterns (masks) and a single-pixel detector. It’s a lifesaver in harsh environments like turbulent atmospheres or low-light conditions where traditional multi-pixel cameras fail.

However, GI faces a classic trade-off: to get a high-resolution image, you either need a massive number of masks (which takes too long for real-time video) or complex "Compressed Sensing" algorithms (which are computationally expensive). The "holy grail" of GI is achieving high-quality reconstruction at "sub-Nyquist" sampling rates—using far fewer patterns than pixels—without sacrificing speed. This paper tackles the challenge of mask ordering: determining which specific patterns should be shown first to capture the most important features of a target object (like a human face) as quickly as possible.

Methods

The authors propose a max-projection method to optimize the sequence of Hadamard masks. Hadamard matrices are popular in GI because they are binary (0s and 1s), making them easy for high-speed Digital Micromirror Devices (DMDs) to display, and they are orthogonal, which simplifies the math.

The core innovation is a "data-driven" approach to ordering:

  1. Training: They take a small set of training images (e.g., faces) and iteratively identify which Hadamard basis vectors (masks) have the maximum projection—essentially the most "overlap" or shared information—with the training set.
  2. Ordering: The masks are ranked based on this contribution. The most "significant" masks are used first during data acquisition.
  3. Fast Reconstruction: Because the masks remain orthogonal, the authors use a simplified Moore-Penrose pseudo-inverse method. This reduces the reconstruction process to a simple matrix multiplication, which is orders of magnitude faster than iterative optimization algorithms.

Datasets

The study primarily used the CelebA dataset, a large-scale face attributes dataset.

  • Training: Surprisingly, the authors found that an excellent ordering could be achieved using a training set of only 20 face images.
  • Evaluation: They tested the algorithm on random face images outside the training set to demonstrate generalization.
  • Target Size: Simulations were performed on $64 \times 64$ pixel images.

Key Insights and Ideas

  • Prior Knowledge is Efficient: You don't need "Big Data" or deep neural networks to improve ghost imaging. By simply knowing the class of the object (e.g., "it's a face"), you can pre-optimize your "questions" (masks) to get the most information out of the first few samples.
  • The Power of 20: The discovery that only 20 training images are needed is a major win for practical applications where data might be scarce. This is because the binary nature of Hadamard masks limits the "search space," making the optimization converge very quickly.
  • Inherent Noise Filtering: The max-projection ordering naturally prioritizes low-frequency components that contribute most to the image structure. At ultralow sampling rates, this acts as a natural filter against high-frequency Gaussian noise, making the method remarkably robust.
  • Speed via Orthogonality: By sticking to orthogonal Hadamard bases rather than random patterns, the reconstruction becomes a "one-shot" matrix operation, enabling millisecond-scale visualization of dynamic scenes.

Key Results

  • Superior Quality: At an ultralow 5% sampling rate, the max-projection ordering achieved a Peak Signal-to-Noise Ratio (PSNR) 1.1 dB higher than the "cake-cutting" method (the previous best-in-class reference).
  • Real-Time Performance: Reconstruction times were reduced to milliseconds, compared to the seconds or minutes required by traditional compressed sensing or deep learning reconstruction methods.
  • Noise Robustness: In noisy simulations (Gaussian white noise), the retrieved images at low sampling rates remained almost identical to noise-free versions, outperforming Walsh and natural ordering schemes.
  • Generalization: Even though the ordering was optimized on a small set, it successfully reconstructed "unseen" faces with high fidelity.

Limitations and Future Work

  • Binarity Constraints: The authors note that the performance of the max-projection method is somewhat restricted by the binary (0, 1) nature of Hadamard masks. They suggest that future work could involve optimizing non-binary but orthogonal masks to capture even more subtle grayscale variations.
  • Class Specificity: While excellent for faces, the ordering is specific to the training class. A "face-optimized" sequence might not perform as well on a "circuit-board-optimized" task.
  • Experimental Validation: The current paper is based on high-fidelity simulations. The authors plan to move toward physical implementation using custom-fabricated gold foil masks for experimental verification.

Summary updated

Abstract

One crucial component of ghost imaging (GI) is the encoded mask. Higher-quality reconstruction at lower sampling rates is still a major challenge for GI. Inspired by deep learning, max-projection method is proposed in the paper to reorder the Hadamard masks for its efficient and rapid reconstruction. The simulations demonstrated that max-projection ordering with only 20 face training images yielded excellent reconstruction outcomes. In noise-free simulations, at an ultralow sampling rate of 5%, the PSNR of the max-projection ordering was 1.1 dB higher than that of the cake-cutting ordering with the best performance in the reference group. In noisy simulations, at ultralow sampling rates, the retrieved images remained almost identical to their noise-free counterparts. Irrespective of the presence or absence of noise, the max-projection ordering guaranteed the highest fidelity of image reconstruction at ultralow sampling rates. The reconstruction time was reduced to mere milliseconds, thereby enabling swift visualization of dynamic phenomena. Accordingly, the max-projection ordering Hadamard matrix offers a promising solution for real-time GI due to its higher reconstruction quality, stronger noise immunity and millisecond reconstruction time.
BibTeX
@inproceedings{zhang2023optimizing,
  author = {Haipeng Zhang and Kang Du and Changzhe Zhao and Jie Tang and Shangyu Si and Wenhong Jia and Lian Xue and Zhong‐Liang Li},
  title = {{Optimizing the ordering of the Hadamard masks of ghost imaging suitable for the efficient face reconstruction using the max-projection method}},
  booktitle = {Scientific},
  year = {2023},
  doi = {10.1038/s41598-023-48453-2},
}
Figure