TL;DR
This paper introduces a novel "per-pixel coded exposure" technique using a Digital Micromirror Device (DMD) camera to achieve high-speed imaging without increasing bandwidth limitations. By cleverly modulating pixel exposure times, they effectively boost the temporal resolution of a standard low-frame-rate camera, demonstrating a 4x increase to 100 fps.
What Problem Does This Paper Address?
Capturing rapid physical phenomena, like shock waves or fluid dynamics, requires high-speed and high-resolution imaging. However, conventional CCD and CMOS cameras are limited by their relatively low frame rates (typically around 30 Hz) and the significant bandwidth required to transfer and store the vast amounts of data generated at higher speeds. This bottleneck prevents them from effectively capturing fast events with sufficient detail. Existing high-speed imaging methods often rely on specialized sensors with parallel readout or in-situ storage, which can be complex and have limitations in sensitivity or frame rate. The authors aim to overcome these bandwidth and frame rate limitations by leveraging a computational imaging approach.
Methods
The core innovation is the "per-pixel coded exposure" method implemented with a DMD camera. Here's how it works:
- DMD as a Programmable Shutter: A Digital Micromirror Device (DMD) is used as a high-speed, pixel-level optical switch. Each micromirror can be individually controlled to deflect light, acting like a tiny, programmable shutter for each pixel.
- Coded Exposure Patterns: Instead of a single, uniform exposure for the entire frame, the DMD projects a sequence of binary coded patterns during a single frame integration time of the CCD camera. These patterns dictate which pixels are exposed and for how long.
- Temporal Information Encoding: By carefully designing these DMD patterns and their timing, temporal information about the scene is encoded into the single, full-resolution image captured by the CCD. Different exposure durations for different pixels within the same frame allow for the reconstruction of multiple "subframes."
- Subframe Extraction: After capturing the coded exposure image, a novel "three-element median quicksort" algorithm is employed. This algorithm efficiently sorts the pixels within each "exposure group" based on their light intensity. This sorting allows for the extraction of multiple subframes, each representing a distinct point in time, effectively increasing the temporal resolution.
- Optical Model and Calibration: The paper details the optical model of the DMD camera and the crucial step of precisely aligning the DMD mirrors with the CCD pixels. This is achieved using a phase-shifting moiré method for accurate correspondence adjustment.
Datasets
Not discussed in the paper. The experiments utilize real-world scenes (candle flame, blowing out candles, liquid mixing) captured by their custom DMD camera setup.
Key Insights and Ideas
- Computational Imaging for Speed: The paper demonstrates that by treating the camera system as a computational imaging device, significant improvements in temporal resolution can be achieved without resorting to faster, more expensive sensors or increased data bandwidth.
- DMD's Versatility: The DMD's ability to act as a highly flexible, per-pixel optical switch is key. It allows for precise control over light modulation at a speed far exceeding traditional mechanical shutters.
- Trade-off Management: The per-pixel coded exposure method allows for a flexible trade-off between spatial and temporal resolution. While the extracted subframes have reduced spatial resolution compared to the original coded image, the temporal resolution is significantly enhanced.
- Efficient Sorting for Reconstruction: The development of the "three-element median quicksort" algorithm is crucial for efficiently processing the large amount of data generated by the coded exposure. This algorithm is designed to be faster than traditional quicksort for this specific application.
Key Results
- 4x Temporal Resolution Increase: The system successfully increased the temporal resolution by a factor of four, achieving 100 frames per second (fps) from a base camera with a 25 fps frame rate.
- Full Resolution Coded Image: The initial capture is a full-resolution image (768x576 pixels), from which lower-resolution high-speed subframes are extracted.
- Effective Demonstration: The method was demonstrated on various dynamic scenes, including candle flames and liquid mixing, showing clear visual results that accurately capture the fast-changing phenomena.
Limitations and Future Work
- DMD Resolution Limit: The physical resolution of the DMD limits the maximum spatial resolution and frame rate achievable by the DMD camera.
- Low-Light Performance: The system's low light efficiency means it may struggle to capture high-quality images in dimly lit conditions.
- Future Applications: The authors suggest potential applications in biomedical imaging (cell activity), high dynamic range imaging, depth-of-field extension, and 3D shape measurement, encouraging further research and development.
Summary updated