TL;DR
This paper introduces a novel imaging technique that uses Single-Photon Avalanche Diodes (SPADs) and Digital Micromirror Devices (DMDs) to overcome "diffraction-photons," a significant source of image degradation in existing systems. By co-designing the illumination patterns and deconvolution algorithms, the researchers can successfully capture high-quality intensity and depth images even in challenging conditions.
What Problem Does This Paper Address?
Imaging systems utilizing SPADs offer high sensitivity and temporal resolution, making them ideal for applications like light-in-flight recording and imaging around corners. However, when these systems employ DMDs for projecting illumination patterns, a problem arises: DMDs have a limited contrast ratio (around 1000:1). This means that even when a mirror is "off," it still reflects some light, leading to "diffraction-photons." These stray photons scatter and illuminate parts of the scene that should be dark, severely degrading the quality of the captured intensity and depth images. Traditional raster-scanning methods also struggle with this issue, often requiring more measurements or sacrificing spatial resolution. This paper aims to tackle this diffraction-photon problem head-on, enabling robust imaging with SPADs and DMDs.
Methods
The core of the proposed solution is a "joint illumination-deconvolution scheme." Instead of treating illumination and image reconstruction as separate steps, they are co-designed.
- Illumination Strategy: The paper proposes using overlapping illumination blocks projected by the DMD. This approach boosts the signal power by collecting more photons per measurement. While this can introduce blur due to photons from adjacent pixels being mixed, it also helps alleviate deadtime-interference in SPADs by concentrating optical power. The size of these illumination blocks is a critical parameter, balancing signal gain against blur.
- Deconvolution Algorithm: To untangle the mixed pixel information caused by the overlapping illumination, a sophisticated deconvolution technique is employed. This involves several steps:
- Variance-Stabilizing Transformation: An Anscombe transform is applied to convert the signal-dependent Poisson noise from the SPAD into approximately additive Gaussian noise, making it easier to process.
- Denoising: A blurred image is denoised using a convex optimization approach (specifically, an L1-norm regularized least-squares method).
- Inverse Transformation: A maximum likelihood transformation is used to correct for biases introduced by the Anscombe transform.
- Deconvolution: Finally, a regularized least-squares approach is used to recover the latent intensity image from the ill-conditioned system.
- Depth Image Reconstruction: For depth imaging, a similar approach is used, but it involves temporal deconvolution. The process includes spatial deconvolution of time-slices, intermediate median filtering, and temporal deconvolution by cross-correlating the received signal with the transmitted pulse waveform to determine the time-of-flight.
- Optimization: Both intensity and depth image reconstruction are formulated as convex optimization problems and solved using the Alternating Direction Method of Multipliers (ADMM) algorithm.
Datasets
The paper does not mention specific publicly available datasets for training or evaluation. Instead, it presents results from a proof-of-concept experiment conducted in a controlled laboratory setting. The "dataset" consists of data collected using their custom SPAD/DMD imaging system with a simple scene (a ball).
Key Insights and Ideas
- Diffraction-Photons as a Major Hurdle: The paper clearly identifies diffraction-photons from low-contrast DMDs as a significant, previously unaddressed challenge in SPAD-based imaging.
- Co-design is Key: The most impactful insight is that a joint approach to illumination and deconvolution is crucial for overcoming diffraction-photons. Simply improving one aspect without considering the other is insufficient.
- Overlapping Blocks for Signal Gain and Deadtime Mitigation: The use of overlapping illumination blocks is a clever strategy to increase photon collection efficiency and reduce SPAD deadtime effects, while the deconvolution handles the resulting spatial mixing.
- ADMM for Robust Reconstruction: The application of ADMM for solving the complex inverse problems in both intensity and depth reconstruction demonstrates its effectiveness in handling ill-conditioned systems and noise.
- Relaxing the Resolution-Signal Trade-off: The proposed scheme offers a way to achieve better image quality without necessarily sacrificing spatial resolution or requiring an excessive number of measurements, a common challenge in such imaging systems.
Key Results
The experimental results demonstrate the effectiveness of the proposed scheme in overcoming diffraction-photons.
- Improved Image Quality: The proposed scheme produces significantly clearer intensity and depth images compared to traditional raster-scanning methods, especially in the presence of diffraction-photons (which were measured to be around 25 photons/pixel).
- Workable Illumination Window Size: An illumination window size of 5x5 pixels was found to provide the best balance between signal photons collected and blur introduced in their experimental setup.
- Comparison with Benchmarks: The paper shows that their method outperforms existing benchmarks ([6] and [7]) when diffraction-photons are present, although it's noted that these benchmarks were designed for environments without such artifacts.
Limitations and Future Work
- Optimal Window Size Determination: The current method for determining the optimal illumination window size relies on experimental testing. Future work could focus on mathematically determining this optimal size based on parameters like contrast ratio, deadtime, and noise level.
- Sensitivity to Noise: The authors suggest analyzing the sensitivity of their deconvolution algorithms (Eqs. 11 and 19) to noise in the system matrix H. This analysis could lead to more robust deconvolution strategies.
- DMD Contrast Ratio: While the scheme overcomes diffraction-photons, it's acknowledged that a higher contrast ratio DMD would inherently reduce the problem. The current work focuses on mitigating the effects of existing low-contrast DMDs.
- Complexity of Optimization: While ADMM is effective, it is computationally intensive. Further research could explore faster optimization techniques.
Summary updated