Chromatic Reconstruction from Monochromatic Event Streams

This direction is split into other directions: color event dataset scarcity. Its last version stays readable and citable here.

Version 1 of 136 papers38 conceptsfrozen 2026-10-04

Event cameras, or Dynamic Vision Sensors (DVS), offer a radical departure from traditional photography. Instead of capturing full frames at fixed intervals, they record asynchronous "events" whenever a pixel detects a change in log-intensity. This allows for microsecond-level temporal resolution and a massive dynamic range. However, this speed comes at a cost: most event sensors are "color-blind," capturing only monochromatic changes in brightness rather than absolute chromatic values.

Bridging this "color gap" is a central challenge in neuromorphic vision. Researchers are developing ways to synthesize high-fidelity color video and 3D scenes by anchoring sparse, monochromatic events to the rich but slow information provided by traditional sensors or generative priors.

The Anchor Problem: Fusing Events with RGB

The most intuitive way to recover color is a hybrid approach: pair a high-speed event camera with a standard RGB camera. In this setup, the RGB frames act as "anchors" that provide the absolute color and texture of the scene, while the event stream fills in the gaps between frames.

As Peiqi Duan and colleagues demonstrate in the EventAid benchmark, this fusion allows for deblurring and super-resolution that exceeds the capabilities of either sensor alone. The intuition is simple: if an RGB frame is blurry due to motion, the event camera provides the precise trajectory of that motion. By "spreading" the color from the blurry frame along the paths defined by the events, systems can reconstruct sharp, high-speed color video.

However, this is mathematically ill-posed. Event cameras record relative changes ($\Delta \log I$), not absolute intensity ($I$). If an object moves into a region where its color was never captured by the RGB sensor, the system has no reference to "anchor" to. Yunfan Lu notes in HR-INR that this often leads to color distortion or "bleeding" when objects move rapidly across complex backgrounds.

Lifting Events to 3D Radiance Fields

The challenge intensifies when moving from 2D video to 3D reconstruction. Frameworks like Dynamic EventNeRF by Viktor Rudnev and EF-3DGS by Bohao Liao attempt to build Neural Radiance Fields (NeRF) or 3D Gaussian Splats (3DGS) using event data.

In these 3D representations, events serve as a high-frequency supervisor. While a standard camera might see a "smear" during a fast camera sweep, the event camera sees a sequence of precise edges. By optimizing the 3D scene so that its projected changes match the observed event stream, researchers can reconstruct geometry with incredible detail. Yet, as Kanghao Chen points out in Elite-EvGS, these models often struggle with "vibrancy." Without absolute color references, the reconstructed 3D textures tend to look desaturated or grayscale, as the events only provide the "structure" of the change, not the "hue" of the surface.

Generative Colorization and Physical Priors

When auxiliary RGB data is unavailable or severely degraded, researchers turn to two alternative strategies: generative "hallucination" and bio-inspired hardware hacks.

  1. Generative Priors: Weilun Li and Lei Sun introduced EvDiff, which uses a diffusion model to reconstruct color video from pure event streams. Instead of trying to calculate color from first principles, the model uses the structural cues in the events to "guess" the most likely colors based on its training on millions of real-world images. This bridges the gap between the sparse, monochromatic signal and a photorealistic output.
  2. Bio-inspired Optics: Perhaps the most creative solution comes from Sami Arja and the team behind Seeing like a Cephalopod. They mimic the eyes of octopuses, which are technically monochromatic but "see" color by exploiting chromatic aberration. By using a lens that blurs different wavelengths of light differently, they encode spectral information into the spatial pattern of the events. A monochromatic sensor can then "decode" these patterns to perceive color without needing a Bayer filter.

Beyond RGB: Multispectral Events

The future of this direction lies in expanding the spectral range. Saad Himmi and Ajad Chhatkuli have pushed event cameras into the multispectral and near-infrared (NIR) domains with MS-EVS. They found that adding spectral diversity—even if it isn't standard "color"—provides a massive boost to tasks like face detection, especially in low-light conditions where traditional RGB sensors fail entirely.

By treating color not just as a visual aesthetic but as a high-dimensional signal to be reconstructed through fusion, priors, or clever optics, this research area is transforming "color-blind" neuromorphic sensors into the most capable high-speed imaging tools available.

Papers in this version

History & citationVersion 1 · 2026-10-04

Split into other directions · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_chromatic_reconstruction_from_monochroma_cdab69fb,
  title = {Chromatic Reconstruction from Monochromatic Event Streams},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/chromatic-reconstruction-from-monochromatic-event-streams/rdr_cdab69fb9eeb4f4d907af70b359b69ae},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 1, frozen text):

@misc{lacuna_chromatic_reconstruction_from_monochroma_cdab69fb_v1,
  title = {Chromatic Reconstruction from Monochromatic Event Streams},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/chromatic-reconstruction-from-monochromatic-event-streams/rdr_cdab69fb9eeb4f4d907af70b359b69ae/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}