Image reconstruction in the gigavision camera

Feng Yang, Luciano Sbaiz, Edoardo Charbon, Sabine Süsstrunk, Martin VetterliWorkshops @ ICCV 2009 · OpenReview

Abstract

Recently we have proposed a new image device called the gigavision camera. The main feature of this camera is that the pixels have a binary response. The response function of a gigavision sensor is non-linear and similar to a logarithmic function, which makes the camera suitable for high dynamic range imaging. Since the sensor can detect a single photon, the camera is very sensitive and can be used for night vision and astronomical imaging. One important aspect of the gigavision camera is how to estimate the light intensity through binary observations. We model the light intensity field as 2D piecewise constant and use Maximum Penalized Likelihood Estimation (MPLE) to recover it. Dynamic programming is used to solve the optimization problem. Due to the complex computation of dynamic programming, greedy algorithm and pruning quadtrees are proposed. They show acceptable reconstruction performance with low computational complexity. Experimental results with synthesized images and real images taken by a single-photon avalanche diode (SPAD) camera are given.

Introduction: The Challenge of Binary Imaging

Conventional digital cameras face two major hurdles: limited dynamic range—the ability to see detail in both deep shadows and bright highlights—and poor low-light sensitivity. Standard sensors convert light into an analog electrical signal that is then digitized. If a scene is too bright, the sensor saturates; if it's too dark, the signal is lost in noise.

The Gigavision sensor addresses this by using a massive array of binary (1-bit) pixels. Each pixel acts as a simple switch: it is either "0" (no photons detected) or "1" (at least one photon detected). By oversampling the scene with billions of these tiny binary pixels, the camera mimics the logarithmic response of the human eye, capturing vast ranges of light intensity without traditional saturation. However, this creates a significant mathematical challenge: reconstructing a high-quality grayscale image from a sea of binary data.

Reconstruction Methodology

The reconstruction process is treated as a statistical estimation problem. The arrival of photons at each pixel is modeled as a Poisson process. To recover the original scene, the authors employ Maximum Penalized Likelihood Estimation (MPLE). This approach finds the most likely light intensity that produced the observed binary pattern while adding a penalty term to prevent overfitting and noise.

The Gigavision reconstruction pipeline: from light intensity field λ(x) to binary observations K, and finally to the reconstructed estimate λ̂(x).

To simplify the optimization, the light intensity is modeled as piecewise constant, assuming the world consists of regions of uniform intensity. This helps preserve sharp edges while reducing computational complexity.

Optimization Algorithms

Three primary algorithms were developed to solve the MPLE problem:

  1. Dynamic Programming (DP): Provides the mathematically optimal solution but is computationally prohibitive for large 2D images ($O(N^9)$ complexity).
  2. Greedy Algorithm: A faster, iterative approach that makes "best-guess" splits to identify uniform regions.
  3. Pruning Quadtrees: The most efficient method for 2D reconstruction. It builds a hierarchical tree structure and "prunes" branches where intensity is uniform, achieving $O(N^2)$ complexity.
Reconstruction of a 1D signal using the pruning binary tree algorithm, demonstrating high fidelity to the original signal.

Key Results and Insights

The Gigavision concept relies on spatial oversampling. By having many more pixels than the final output resolution, the binary states can be averaged to create smooth grayscale transitions. Because the probability of a pixel staying "0" decreases exponentially as light increases, the sensor naturally behaves like a logarithmic detector, enabling High Dynamic Range (HDR) imaging in a single exposure.

In simulations, the Quadtree algorithm achieved PSNR values between 28dB and 32dB, providing visual results nearly identical to the optimal DP solution but at a fraction of the speed. Real-world validation was performed using a Single-Photon Avalanche Diode (SPAD) camera, successfully turning unrecognizable binary noise into clear images of laboratory scenes.

Future Work

While the current piecewise constant model is effective, future research may explore piecewise linear or piecewise smooth models to better represent curved surfaces and gradients. Additionally, as hardware catches up to the "gigavision" scale, implementing these $O(N^2)$ algorithms on specialized hardware like FPGAs will be crucial for real-time applications.

Summary updated

Abstract

Recently we have proposed a new image device called the gigavision camera. The main feature of this camera is that the pixels have a binary response. The response function of a gigavision sensor is non-linear and similar to a logarithmic function, which makes the camera suitable for high dynamic range imaging. Since the sensor can detect a single photon, the camera is very sensitive and can be used for night vision and astronomical imaging. One important aspect of the gigavision camera is how to estimate the light intensity through binary observations. We model the light intensity field as 2D piecewise constant and use Maximum Penalized Likelihood Estimation (MPLE) to recover it. Dynamic programming is used to solve the optimization problem. Due to the complex computation of dynamic programming, greedy algorithm and pruning quadtrees are proposed. They show acceptable reconstruction performance with low computational complexity. Experimental results with synthesized images and real images taken by a single-photon avalanche diode (SPAD) camera are given.
BibTeX
@inproceedings{yang2009image,
  author = {Feng Yang and Luciano Sbaiz and Edoardo Charbon and Sabine Süsstrunk and Martin Vetterli},
  title = {{Image reconstruction in the gigavision camera}},
  booktitle = {Workshops @ ICCV},
  year = {2009},
  url = {https://openreview.net/forum?id=KY-hg77szOG},
}
Figure