QR-code Reconstruction from Event Data via Optimization in Code Subspace

Jun Nagata, Yusuke Sekikawa, Kosuke Hara, Teppei Suzuki, Yoshimitsu AokiWACV 2020 · OpenReview

Abstract

Reconstructing QR codes from high-speed motion is a significant challenge for traditional frame-based cameras due to motion blur. This paper proposes a novel method using event-based cameras, which record per-pixel intensity changes with microsecond precision. By optimizing within a constrained "code subspace" that leverages the known structure of QR codes, the method achieves robust reconstruction and high readability in scenarios where conventional high-speed industrial cameras fail. Experimental results on a new benchmark dataset demonstrate a jump from 0% to over 90% readability in high-speed industrial environments.

The Challenge: Motion Blur in Industrial Automation

In factory automation and logistics, QR codes are frequently attached to objects moving at high speeds, such as on belt conveyors. Standard frame-based cameras suffer from severe motion blur in these scenarios, rendering the codes unreadable. While specialized high-speed industrial cameras (e.g., 330 fps) can mitigate this, they are expensive, power-hungry, and generate massive amounts of redundant data.

Motion blur in a standard frame-based camera capturing a moving QR code makes it unreadable.

Event-based cameras (neuromorphic sensors) offer a compelling alternative. Unlike traditional cameras, they only record changes in intensity per pixel. This sparse, high-temporal-resolution data effectively eliminates motion blur. However, reconstructing a clean image from this "stream of events" is a mathematically ill-posed problem due to the sparsity and noise inherent in the signal.

Methodology: Optimization in Code Subspace

The authors' key innovation is Optimization in Code Subspace. Rather than attempting to reconstruct every pixel in a standard image space (e.g., $480 \times 640$), they restrict the solution to a "code space" defined by valid QR code structures and affine transformations (rotation, scaling, shearing).

1. Parameter Reduction

By assuming the target is a QR code, the number of variables is reduced from hundreds of thousands of pixels to a small $r \times r$ grid of QR bits, a 2D motion vector, and six affine transformation parameters. This massive reduction in the search space acts as a powerful regularizer, allowing the algorithm to ignore noise that would typically degrade general-purpose reconstruction.

2. Two-Step Optimization Process

  • Initial Estimation: The system first estimates the initial position and motion using the QR code's unique "finder patterns" (the 1:1:3:1:1 ratio corner squares). Principal Component Analysis (PCA) in spacetime is used to derive an initial velocity.
  • Bit Optimization: Once the geometry is roughly aligned, the individual bits ($q$) of the QR code are optimized to match the event data. The researchers utilize a differentiable bilinear interpolation method to link discrete QR bits to the continuous intensity changes recorded by the sensor, enabling the use of standard gradient descent.
The event generation principle (a) and a comparison between the sparse event stream and a blurred traditional frame (b).

Experimental Results and Benchmarking

The researchers introduced the QR-code Event Camera Dataset (QRECD), featuring both simulated data (via ESIM) and real-world captures using a Samsung DVS Gen3 event camera. In tests involving a toy train carrying QR codes at speeds up to 2m/s, the proposed method significantly outperformed existing technologies:

  • Readability: In high-speed scenes (2m/s), a 330-fps industrial camera produced blurred images with a 0% read rate. The proposed method achieved a 90-99% read rate.
  • Bit-Level Accuracy: The method maintained nearly 100% bit-level accuracy across most tested scenes, even when the QR code was tilted at a 45-degree angle.
  • Efficiency: By constraining the parameter space, the reconstruction is faster and more robust to the high noise levels typical of event sensors.

Limitations and Future Directions

While highly effective, the current model relies on an affine transformation, which assumes the QR code is on a flat surface viewed from a distance. Future work will involve extending the model to projective transformations to handle significant handheld tilts. Additionally, the method is class-specific; while the principle of subspace optimization could be applied to other known geometries (like license plates), it is not intended as a general-purpose image restoration tool. Finally, the optimization's success is sensitive to the initial detection of finder patterns, a dependency that remains a focus for further refinement.

Summary updated

Abstract

We propose an image reconstruction method from event data, assuming the target images belong to a prespecified class like QR codes. Instead of solving the reconstruction problem in the image space, we introduce a code space that covers all the noiseless target class images and solves the reconstruction problem on it. This restriction enormously reduces the number of optimizing parameters and makes the reconstruction problem well posed and robust to noise. We demonstrate fast and robust QR-code scanning in difficult, high-speed scenes with industrial high-speed cameras and other reconstruction methods.
BibTeX
@inproceedings{nagata2020qr,
  author = {Jun Nagata and Yusuke Sekikawa and Kosuke Hara and Teppei Suzuki and Yoshimitsu Aoki},
  title = {{QR-code Reconstruction from Event Data via Optimization in Code Subspace}},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision},
  year = {2020},
  url = {https://openreview.net/forum?id=Qn0r0tRXi-},
}
Figure