Abstract
Reconstructing QR codes from high-speed motion is a significant challenge for traditional frame-based cameras due to motion blur. This paper proposes a novel method using event-based cameras, which record per-pixel intensity changes with microsecond precision. By optimizing within a constrained "code subspace" that leverages the known structure of QR codes, the method achieves robust reconstruction and high readability in scenarios where conventional high-speed industrial cameras fail. Experimental results on a new benchmark dataset demonstrate a jump from 0% to over 90% readability in high-speed industrial environments.
The Challenge: Motion Blur in Industrial Automation
In factory automation and logistics, QR codes are frequently attached to objects moving at high speeds, such as on belt conveyors. Standard frame-based cameras suffer from severe motion blur in these scenarios, rendering the codes unreadable. While specialized high-speed industrial cameras (e.g., 330 fps) can mitigate this, they are expensive, power-hungry, and generate massive amounts of redundant data.
Event-based cameras (neuromorphic sensors) offer a compelling alternative. Unlike traditional cameras, they only record changes in intensity per pixel. This sparse, high-temporal-resolution data effectively eliminates motion blur. However, reconstructing a clean image from this "stream of events" is a mathematically ill-posed problem due to the sparsity and noise inherent in the signal.
Methodology: Optimization in Code Subspace
The authors' key innovation is Optimization in Code Subspace. Rather than attempting to reconstruct every pixel in a standard image space (e.g., $480 \times 640$), they restrict the solution to a "code space" defined by valid QR code structures and affine transformations (rotation, scaling, shearing).
1. Parameter Reduction
By assuming the target is a QR code, the number of variables is reduced from hundreds of thousands of pixels to a small $r \times r$ grid of QR bits, a 2D motion vector, and six affine transformation parameters. This massive reduction in the search space acts as a powerful regularizer, allowing the algorithm to ignore noise that would typically degrade general-purpose reconstruction.
2. Two-Step Optimization Process
- Initial Estimation: The system first estimates the initial position and motion using the QR code's unique "finder patterns" (the 1:1:3:1:1 ratio corner squares). Principal Component Analysis (PCA) in spacetime is used to derive an initial velocity.
- Bit Optimization: Once the geometry is roughly aligned, the individual bits ($q$) of the QR code are optimized to match the event data. The researchers utilize a differentiable bilinear interpolation method to link discrete QR bits to the continuous intensity changes recorded by the sensor, enabling the use of standard gradient descent.
Experimental Results and Benchmarking
The researchers introduced the QR-code Event Camera Dataset (QRECD), featuring both simulated data (via ESIM) and real-world captures using a Samsung DVS Gen3 event camera. In tests involving a toy train carrying QR codes at speeds up to 2m/s, the proposed method significantly outperformed existing technologies:
- Readability: In high-speed scenes (2m/s), a 330-fps industrial camera produced blurred images with a 0% read rate. The proposed method achieved a 90-99% read rate.
- Bit-Level Accuracy: The method maintained nearly 100% bit-level accuracy across most tested scenes, even when the QR code was tilted at a 45-degree angle.
- Efficiency: By constraining the parameter space, the reconstruction is faster and more robust to the high noise levels typical of event sensors.
Limitations and Future Directions
While highly effective, the current model relies on an affine transformation, which assumes the QR code is on a flat surface viewed from a distance. Future work will involve extending the model to projective transformations to handle significant handheld tilts. Additionally, the method is class-specific; while the principle of subspace optimization could be applied to other known geometries (like license plates), it is not intended as a general-purpose image restoration tool. Finally, the optimization's success is sensitive to the initial detection of finder patterns, a dependency that remains a focus for further refinement.
Summary updated