Dropouts as Measurements: Physics-Censored Tracking for Range-Limited Active Sensing

Active optical sensors such as flash cameras, gated cameras, time-of-flight units and blinking-LED trackers work by emitting light and measuring what comes back. Most of them cancel sunlight and room light by subtracting a frame with the emitter off from a frame with it on. Two effects then limit range. The returned light shrinks with the fourth power of distance for reflected light, or the square for a self-luminous marker. Subtraction also removes the average ambient light but leaves its random photon noise behind. Past some distance the signal falls below a detection threshold and the sensor reports nothing. Trackers built on these sensors treat those gaps as missing data. They coast on a motion model, fill the gaps, or give up.

This proposal starts from the observation that these gaps carry information. A non-detection at a known emitter power and a measured ambient level means that the target is probably farther away, darker, or more backlit than a threshold the physics lets us compute. We propose to write that fact into the tracking objective as a censored observation, meaning a measurement that only says "the value was below the threshold." We then fit the target's continuous-time trajectory to detections and non-detections together. The claim is that this physics-derived treatment of missing data cuts position error and overconfidence during range-induced dropouts, and that it transfers across emitter power and ambient light levels in ways that learned, missingness-aware sequence models cannot.

Core Claim

The irregular-time-series literature already argues that the pattern of missing observations is informative. MTM aligns channels that are sampled out of sync by pooling across temporal scales without imputing. It argues that imputation destroys the signal carried by the sampling pattern, and it reports gains of up to 3.8% AUPRC on the PhysioNet 2012 clinical benchmark. In clinical data, why an observation is missing has to be learned, because nobody has a formula for when a doctor orders a test. In active optical sensing the cause is known. The dropout probability follows from the inverse-power falloff, the emitter power and the ambient shot noise.

With on/off subtraction, write the differential signal as $S(r) = \kappa P \rho / r^{n}$, where $r$ is the target distance, $P$ the emitter power, $\rho$ the target reflectance, $\kappa$ a calibration constant, and $n=4$ for reflected light or $n=2$ for an emitting marker. The subtracted pixel then has noise variance $\sigma^2 = S + 2I_a + 2\sigma_r^2$. Here $I_a$ is the ambient photon count, which appears in both frames, and $\sigma_r$ is the read noise. The probability of detecting the target at a threshold $\tau$ is therefore

$$ p_d(r, I_a) = \Phi\!\left(\frac{S(r) - \tau}{\sigma}\right), $$

where $\Phi$ is the standard normal cumulative distribution. For a latent trajectory $x(t)$ with distance $r(t)$, the objective sums two kinds of terms. For each detection time $t_k$ it adds the ordinary measurement likelihood. For each non-detection time $t_j$ it adds a censored term:

$$ \log L = \sum_{k \in \text{det}} \log p(y_k \mid x(t_k)) + \sum_{j \in \text{miss}} \log\big[(1-\pi_o)(1 - p_d(r(t_j), I_{a,j})) + \pi_o\big], $$

where $\pi_o$ is the probability that the dropout was caused by occlusion. The occlusion term stops the model from reading every blocked view as "far away."

The censored term pushes the estimate toward trajectories that would plausibly have gone unseen. This is the information that coasting filters and imputation both throw away. Because $I_a$ is measured in every off-frame, the model also knows that the same dropout means something different at noon than at night.

Relation to Prior Work

Three lines of work motivate the proposal by showing that range dropouts are the binding failure. The event-camera active LED marker system tunes the sensor's internal settings so the hardware acts as a band-pass filter that rejects steady ambient light, and it tracks 6-DoF pose at 3.8 kHz with 0.35 ms latency. Tracking fails beyond roughly 10 m, where the marker's flux drops below the camera's contrast threshold. Raw time-of-flight tracking at 300 Hz skips depth reconstruction and tracks from the raw measurements, but loses the signal beyond about 10 m. Flash-only reflection removal and its handheld extension use on/off subtraction in raw sensor space, and the cue disappears for distant scenes. All three treat the loss of signal as an endpoint. None uses it as evidence.

Other work widens the operating range with hardware and constrains our design. Patterned flash concentrates power into sparse dots. Light curtains and their rolling-shutter version steer power and interleave laser-on and laser-off exposures, working under 100 klx and 50 klx of sunlight respectively. Gated Stereo adds wide-baseline stereo to reach 160 m within eye-safety limits. These are complementary: each moves the detection threshold, and our method works at whatever threshold the hardware leaves. The eye-safety caps they report are the reason we cannot simply raise emitter power.

The estimator comes from continuous-time sequence models. Neural controlled differential equations represent a hidden state that evolves in continuous time, driven by irregular observations. The continuous-time autoencoder (CTA) uses this approach for imputation and stays stable at 70% missingness with about 0.02M parameters. We reuse this machinery for irregular event and frame timestamps. Unlike those works, we add the censored likelihood; their imputation treats a missing value as uninformative.

Research Program

We will build a single estimator with three parts. The first is a calibrated detection model $p_d$, fitted per sensor from controlled captures at known distances, reflectances and ambient levels. This checks the Gaussian-threshold form above against what the sensor actually does. For event cameras we expect the form to need replacing with a contrast-threshold model, and we will report that.

The second is a latent trajectory model. We will implement it twice, as a particle filter with a physical motion prior and as a neural CDE trained end to end with the censored objective. This separates the value of the censoring term from the value of learning.

The third is an explicit occlusion term $\pi_o$, either fixed or predicted from image context. We will analyze when the censored term is identifiable. Because of the fourth-power falloff, a non-detection mainly limits the product $\rho / r^{4}$. It therefore constrains distance only when reflectance has been estimated from earlier detections. We expect the gains to concentrate in the period right after a target leaves detection range, and we will measure how that benefit decays with time since the last detection.

Baselines

The naive reference holds the last detected position and inflates its uncertainty over time. It shows how much any modeling helps at all.

The standard engineering baseline is a constant-velocity Kalman filter that skips updates during dropouts. It is fast and widely deployed, and it is the method a reviewer would expect us to beat. It gets the same detections, timestamps and motion-noise tuning budget as our method.

The strongest continuous-time competitor is the CTA imputer from Continuous-time Autoencoders, which reports the lowest MAE and RMSE against 19 baselines on AirQuality, Stocks, Electricity and Energy. We will retrain it on our trajectories with an equal hyperparameter search.

The strongest missingness-aware competitor is an MTM-style model that receives the observation mask, and also $I_a$ and emitter power as input features. It must learn the dropout mechanism from data. This is the key ablation. It has the same information as our method, and differs only in having no physics prior.

For the hardware setting, the reference numbers are the ALM system's reported 3.8 kHz rate and 0.35 ms latency. No reported accuracy result was found on Lacuna for tracking beyond 10 m. For depth, Gated Stereo reports 2.25 m MAE in both day and night on its dataset, against 9.51 m for monocular gated methods in daylight. Beating a baseline means lower position RMSE and better-calibrated uncertainty during dropout intervals, with three-seed confidence intervals that do not overlap, and no loss on frames where the target is detected.

Why not just…?

Why not just raise emitter power? Eye-safety rules cap peak near-infrared power, which is why Gated Stereo combines active cues with stereo. Light curtains show that bright ambient light fills pixel wells before any active signal is recorded. More power only moves the threshold. Dropouts remain at the new boundary, and our method applies there unchanged.

Why not just use a Kalman filter that coasts through gaps? A coasting filter's uncertainty grows equally in every direction, and its estimate stays centered on the extrapolated path. It cannot use the fact that the target became invisible. On a target receding at constant speed the two approaches may agree. They should differ when the target turns or decelerates during the dropout, or when ambient light changes. Our synthetic benchmark includes those cases on purpose. If the coasting filter matches us there, the claim fails.

Why not just let a mask-aware network learn the dropout mechanism? This is the real threat. A network given the mask, $I_a$ and emitter power could learn $p_d$ within the conditions it was trained on. Our bet is extrapolation. We will train at one emitter power and ambient range and test at others, such as night-trained models evaluated in daylight or at a different power. If the learned model matches ours out of distribution, the contribution shrinks to a data-efficiency result, and we will say so.

Why not just use a faster frame camera? High-speed subtraction produces more measurements, but at shorter exposures, so each one sits closer to the read-noise floor. The per-frame detection limit gets worse, and dropouts become more frequent rather than rarer. Our method uses those extra non-detections as additional censored terms rather than discarding them.

Proposed Experiments

The first experiment is semi-synthetic. We will render FlyingThings3D sequences with physically simulated flash-on and flash-off frames, using fourth-power falloff and shot-plus-read noise in the style of the patterned-flash simulation. Ambient levels will be swept from dark to daylight equivalents, and objects will be scripted to leave and re-enter detection range, including turns made while undetected. We will measure position RMSE during dropouts, negative log-likelihood, 90% interval coverage, and error at the moment of re-detection.

The second is a hardware test that recreates the ALM setup. A marker will travel on a rail from 3 to 25 m, indoors and outdoors, with total-station ground truth, and we will measure pose error beyond the reported 10 m limit.

The third applies the censored term per object to far-range bins of the Gated Stereo dataset. There, a missing gated return combined with the measured ambient slice bounds depth. We will test whether adding this term to Gated Stereo's stereo-based depth reduces MAE on objects beyond 80 m. If the release lacks consecutive frames, we will use it only to calibrate $p_d$.

The decisive check is the out-of-distribution comparison between the physics-censored models and the mask-aware learned model, reported as a function of time since the last detection.