Event-Camera Denoising with a Guaranteed Noise Fraction

An event camera reports a stream of per-pixel "something changed" events instead of full frames. In dim light most of those events are spurious background activity caused by sensor leakage. Every spurious event that reaches a spiking neural network (a network whose neurons communicate with sparse binary pulses, so its energy scales with how many pulses fire) causes extra firing, extra energy use and lower accuracy. The practical question is therefore a statistical one. For each incoming event the filter must decide whether it is noise, the event stream never ends, and the filter cannot spend more than a handful of additions per event.

We propose to treat that decision as online multiple hypothesis testing. Each event carries a null hypothesis, "this event is background noise." A cheap p-value comes from its spatio-temporal neighborhood, and events are passed only when an online false discovery rate (FDR) rule rejects the null. The FDR is the expected fraction of passed events that are actually noise. Under the assumptions laid out below, this fraction is bounded at a user-chosen level $\alpha$ whatever the illumination or scene content. The online rule also has a useful hardware property. Its running significance budget grows, is spent and leaks away in the same way as a leaky integrate-and-fire neuron's membrane potential, so it could run on neuromorphic hardware as one extra neuron per region. The result would be the first event filter with a stated error guarantee, and one whose downstream spike energy can be predicted from $\alpha$.

Core Claim

Current denoisers pick an operating point empirically. The 4-bit multilayer perceptron filter of Within-Camera Multilayer Perceptron DVS Denoising classifies each event with a threshold fixed during supervised training. The noise-filtering neurons of Bio-Vision-Inspired Spiking Neural Networks for Object Detection with Event Cameras raise and lower a soft threshold heuristically. Neither says what fraction of the surviving stream is noise. That fraction should drift badly when light levels change, because background activity rises in the dark while real signal falls. We claim that an online FDR filter keeps the noise fraction of the passed stream at or below $\alpha$ across the illumination levels in E-MLB: Multilevel Benchmark for Event-Based Camera Denoising, and that it does so at a per-event cost comparable to classic neighbor-count filters. We further claim that holding purity fixed makes downstream accuracy and spike count more stable under illumination shift than any fixed threshold. The novelty is not denoising itself. It is (a) a guarantee on the quantity that matters downstream, namely purity of the passed stream, and (b) an online rule that runs as a leaky integrator.

Research Program

P-values from free null data. Conformal p-values need a calibration set of known nulls drawn exchangeably from the null distribution, which is the same assumption High-Power Training Data Identification with Provable Statistical Guarantees relies on and flags as a limitation. Event cameras make such data unusually easy to get. A fixed camera looking at a static scene sees no brightness changes, so everything it emits is background activity, at the same illumination and temperature as deployment. The procedure has three steps:

  1. Record short static clips at each illumination level.
  2. For each event, compute a score: the count of recent events in a small space-time window around it, weighted by how old those events are.
  3. Turn the score into a p-value by ranking it against the scores of calibration noise events.

We will also compare against a parametric p-value that assumes neighbor counts follow a Poisson distribution at a locally estimated noise rate. This version needs no calibration data, but its guarantee holds only if the Poisson model and the rate estimate are correct.

An online rule that runs as a leaky integrator. We adopt generalized alpha-investing with decaying memory from Online control of the false discovery rate with decaying memory. Each region keeps a significance budget $W_t$ (its "wealth"). Event $t$ is tested at level $\alpha_t \le W_t$, and the budget evolves as

$$W_{t+1} = \delta\, W_t - \alpha_t + R_t\, \psi_t,$$

where $\delta \in (0,1)$ is the memory decay, $R_t \in \{0,1\}$ indicates that event $t$ was passed, and $\psi_t$ is the payout the rule grants for a rejection. The decay prevents two failure modes. Without it, the filter can run out of budget during long dark stretches ("alpha-death"). It can also use old discoveries to justify passing weak events now ("piggybacking"), which here would mean passing noise shortly after a burst of real motion.

Read as a circuit, the update is a leaky integrator. Rejections inject current, tests drain it, and $\delta$ is the leak. We will implement it as one neuron per spatial tile, using only additions and comparisons. We will count its operations alongside those of the downstream spiking network. Tiles share no budget, so the error-control argument applies tile by tile.

Dependence. This is the main theoretical risk. Neighboring events share neighborhoods, so their p-values are dependent. Conformal p-values that share one calibration set are also dependent. The decaying-memory guarantee was proved for independent p-values. We will pursue two routes in parallel:

  1. A conservative spending schedule, of the kind usually used when dependence is arbitrary. Its guarantee should survive dependence, but it may cost power.
  2. The standard spending schedule, with realized noise fraction measured empirically wherever ground-truth labels exist.

The negative-dependence results in Multiple testing under negative dependence and the online conformal-selection analysis in Feedback-Enhanced Online Multiple Testing with Applications to Conformal Selection suggest which dependence structures are harmless. We will report which route's guarantee actually holds, rather than assuming one.

Downstream energy model. Following Going Deeper With Directly-Trained Larger Spiking Neural Networks and the energy accounting in Bio-Vision-Inspired Spiking Neural Networks for Object Detection with Event Cameras, we will estimate the network's energy from accumulate operations. These are the additions a spiking neuron performs in place of multiply-accumulates, and their number is proportional to spike count. The filter's own operations are added to that total.

Baselines

Unfiltered stream. The do-nothing reference feeds all events into the same spiking detector. It sets the ceiling on energy and shows the accuracy cost of noise.

Classic neighbor-count filter with a fixed threshold. This is a background-activity filter of the kind benchmarked in E-MLB. It is the cheapest serious competitor and has the same per-event cost as our method, which makes it the fairest test of whether the multiplicity rule adds anything. Its threshold will be tuned on the same validation illumination level our method uses for calibration.

Learned per-event classifier. This is the multilayer perceptron filter of Within-Camera Multilayer Perceptron DVS Denoising, reimplemented in software with 4-bit weights and trained on the same labeled data. Its hardware version reports 4 nJ per event at up to 25 MHz in 65 nm. We will match its input window to ours.

Denoising inside the detector. The strongest competitor overall is STATNF-BiSNet from Bio-Vision-Inspired Spiking Neural Networks for Object Detection with Event Cameras, which reports 69.1% mAP₅₀ on Gen1 and 4.3× better energy efficiency than the previous best spiking network. The comparison runs as follows:

  1. Train the same spiking detector on clean Gen1 data under an identical epoch and tuning budget.
  2. Evaluate it on Gen1 with injected noise, with our filter in front and with no filter.
  3. Evaluate STATNF-BiSNet on the same noisy data.

What counts as winning. On in-distribution illumination, we must match these baselines' mAP at equal or lower total accumulate-operation count. At illumination levels not seen during tuning, we must keep the measured noise fraction at or below $\alpha$ while the fixed-threshold and learned baselines exceed it.

Ablations. We will run the identical p-values through three rules:

  1. A fixed per-event level $\alpha$, which controls the false alarm rate on noise events but not the fraction of passed events that are noise.
  2. The online rule without decay.
  3. Batch Benjamini–Hochberg over one-millisecond buffers.

Comparing these isolates whether the gain comes from the online FDR rule rather than from the p-values. Replacing conformal p-values with parametric Poisson ones isolates the role of calibration.

Why not just…?

Why not just tune a neighbor-count threshold per illumination level? If the illumination level is known and the scene's signal density is stable, a tuned threshold could perform as well as our method. This is a real threat. A fixed threshold controls how often noise passes, but the fraction of the output that is noise depends on how much real signal is present. That fraction should therefore climb in sparse or dark scenes even at a matched noise rate. The fixed-per-event ablation settles this directly. If its noise fraction stays flat across E-MLB scenes of varying content, the multiplicity rule is unnecessary, and we would report that.

Why not just use the learned filter, which already removes noise well? E-MLB finds that deep denoisers preserve structure better than classic filters. However, the multiperceptron filter's threshold comes from training data at particular noise levels, and nothing bounds its error when conditions move. In our experiments we will train it at one illumination level and test it at the others. If its noise fraction stays at or below $\alpha$, the argument for guarantees weakens to the case of never-seen sensors. There is also a middle path worth testing. The learned filter's output can serve as the conformal score, which combines its power with our guarantee.

Why not just buffer a short window and run Benjamini–Hochberg? Batch FDR over millisecond windows is well understood and handles dependence reasonably well. Its costs are that it needs sorting and buffering, which add latency and memory, the very resources in-camera designs are short of. The batch ablation will quantify how much power the online rule gives up in exchange for no buffering.

Proposed Experiments

Test 1: noise-fraction control. We will need labeled noise, and E-MLB supplies no per-event labels; it scores denoising with its Event Structural Ratio. We will therefore inject synthetic background activity at known rates into well-lit Gen1 and DVS-Gesture recordings, and use static-scene recordings as pure-null streams. We will plot the realized noise fraction against target $\alpha$ across injected rates and illumination levels. The success criterion is a curve at or below the diagonal for our method under at least one dependence route.

Test 2: natural data. On E-MLB we will report its structural ratio at each illumination level. The file gives no Event Structural Ratio values for the multiperceptron filter or for EDnCNN on E-MLB, so every method will be run under the same protocol.

Test 3: downstream detection. On Gen1 and N-Caltech101 with injected noise, we will plot mAP₅₀ against total accumulate operations (filter plus network) as $\alpha$ sweeps. The decisive outcome is that the $\alpha$ setting chosen at one illumination level transfers to others with predictable spike count and mAP. The tuned baselines are expected to drift.

Test 4: the hardware claim. We will count per-event memory and operations for the budget neuron. This addresses the SRAM-dominated area reported for in-camera filtering, and it is a necessary check on the claim that the rule fits on neuromorphic hardware.