Motion-Blur-Free High-Speed Video Shooting Using a Resonant Mirror

Michiaki Inoue, Qingyi Gu, Mingjun Jiang, Takeshi Takaki, Idaku Ishii, Kenji TajimaSensors 2017 · DOI · OpenReview

TL;DR

This paper introduces an "actuator-driven frame-by-frame intermittent tracking" system that uses a resonant mirror to achieve motion-blur-free high-speed video of fast-moving objects. By synchronizing camera shutter timings with the mirror's high-frequency vibration, the system captures clear images at 750 fps with a 0.33 ms exposure time, overcoming limitations of previous methods.

What Problem Does This Paper Address?

Capturing clear, high-speed video of fast-moving objects presents a significant challenge due to motion blur. To mitigate this, high-speed cameras typically require shorter exposure times. However, this leads to darker images, especially in low-light conditions or when using high magnification, creating a trade-off between image brightness and motion blur. Existing motion deblurring techniques focus on restoring already blurred images, but they cannot prevent blur from occurring in the first place. Previous attempts at motion-blur-free shooting often involved complex mechanical systems or limited the camera's frame rate, failing to fully leverage the capabilities of high-speed vision systems. This paper aims to solve this by enabling motion-blur-free capture of fast-moving objects without reducing exposure time, thus maintaining image brightness.

Methods

The core innovation is the actuator-driven frame-by-frame intermittent tracking method. Unlike previous "camera-driven" approaches where actuators followed camera frame timings, this new method synchronizes the camera's shutter timings with the resonant vibration of a free-vibration-type actuator, specifically a resonant mirror.

Here's how it works:

  • Resonant Mirror: A resonant mirror is used, vibrating at a high frequency (750 Hz in their prototype). This vibration creates a sinusoidal trajectory for the camera's viewpoint.
  • Synchronized Shutter Control: The camera's exposure is timed to occur only during the highly linear portion of the mirror's sinusoidal trajectory. This ensures that the camera's viewpoint moves at a constant speed relative to the scene during the exposure.
  • Motion Blur Cancellation: By matching the camera's apparent motion to the object's speed during the brief exposure window, motion blur is effectively canceled.
  • Parameter Optimization: The paper details how to determine optimal parameters, such as the temporal aperture ratio (exposure time / cycle time) and amplitude, to minimize the squared-error loss between the mirror's trajectory and the target object's motion. This ensures the best possible motion blur reduction.

This approach leverages the high-frequency response of the actuator to achieve faster tracking and higher frame rates than previous methods, which were limited by actuator response times and movable ranges.

Datasets

Not discussed in the paper. The experiments used custom setups with patterned objects (circle-dots, checkered patterns, electronic board patterns, book pages) moved on a high-speed belt-conveyor system.

Key Insights and Ideas

  • Actuator-Driven Synchronization is Key: The fundamental shift from camera-driven to actuator-driven synchronization is crucial. By letting the actuator's natural resonance dictate the timing, the system can exploit its high-frequency capabilities more effectively.
  • Exploiting Linearity in Sinusoidal Motion: The method cleverly uses the linear segments of the resonant mirror's sinusoidal trajectory. By timing the camera exposure to these segments, the system achieves a constant apparent velocity, canceling out motion blur.
  • Trade-offs in Actuator Design: The paper highlights inherent trade-offs in high-frequency actuators: limited movable range and controllability at high frequencies due to resonance and hysteresis. The proposed method aims to maximize performance within these constraints.
  • Quantitative Blur Reduction: The introduction of blur indexes (Adot, Aedge) allows for objective measurement and demonstration of the effectiveness of the proposed method in reducing motion blur across various scenarios.

Key Results

The developed system successfully captured 1024 × 1024 images at 750 fps with an exposure time of 0.33 ms without motion blur.

  • Motion Blur Reduction: Experiments showed significant reduction in motion blur for various patterned objects moving at speeds up to 7.55 m/s. For example, blur indexes for a circle-dot pattern were remarkably low with the proposed method compared to no-tracking (NT) images.
  • Speed and Resolution: The system achieved high-speed capture (750 fps) at a high resolution (1024 × 1024 pixels) with a short exposure time (0.33 ms).
  • Brightness Maintained: The method allows for motion-blur-free shooting without reducing exposure time, thus preserving image brightness.
  • Complex Patterns Observable: The system demonstrated its ability to capture clear images of complex patterns, such as fine wiring on an electronic board and text on a book page, even at high speeds.

Limitations and Future Work

  • Time-Varying Speeds: The current one-DOF mirror system has limitations in shooting motion-blur-free videos of objects moving at rapid, time-varying speeds in variable directions. The dynamic response of the amplitude control is slow compared to the mirror's resonant frequency.
  • Mirror Size and Aperture: The efficacy in obtaining incident light is practically limited by the mirror size, especially when using very large aperture lenses for zooming.
  • Future Work: The authors plan to extend the system to a two-DOF mirror system for independent pan and tilt control. They also aim to improve it by feedback-controlling camera frame timings and vibration amplitudes using high-speed real-time video processing to estimate object speed and compute blur indexes. Applications in factory automation and vehicle-based infrastructure inspection are envisioned.

Summary updated

Abstract

This study proposes a novel concept of actuator-driven frame-by-frame intermittent tracking for motion-blur-free video shooting of fast-moving objects. The camera frame and shutter timings are controlled for motion blur reduction in synchronization with a free-vibration-type actuator vibrating with a large amplitude at hundreds of hertz so that motion blur can be significantly reduced in free-viewpoint high-frame-rate video shooting for fast-moving objects by deriving the maximum performance of the actuator. We develop a prototype of a motion-blur-free video shooting system by implementing our frame-by-frame intermittent tracking algorithm on a high-speed video camera system with a resonant mirror vibrating at 750 Hz. It can capture 1024 × 1024 images of fast-moving objects at 750 fps with an exposure time of 0.33 ms without motion blur. Several experimental results for fast-moving objects verify that our proposed method can reduce image degradation from motion blur without decreasing the camera exposure time.
BibTeX
@article{inoue2017motion,
  author = {Michiaki Inoue and Qingyi Gu and Mingjun Jiang and Takeshi Takaki and Idaku Ishii and Kenji Tajima},
  title = {{Motion-Blur-Free High-Speed Video Shooting Using a Resonant Mirror}},
  journal = {Sensors},
  year = {2017},
  doi = {10.3390/s17112483},
}
Figure