Time-Honest Simulation for Adaptive Particle Filters

A particle filter tracks a hidden quantity, such as a robot's position, with a cloud of weighted guesses called particles. Adaptive versions grow the cloud when the filter is uncertain and shrink it when it is confident. These adaptive filters are now mostly tuned and compared in simulators, and simulators make two convenient simplifications. First, they wait for the filter to finish before producing the next sensor reading, so computation is free. Second, they generate data from the same motion and sensor models the filter assumes, so the filter's model is exactly right. On a real robot neither holds. Readings keep arriving while the filter computes, and the models are always somewhat wrong. Twenty years ago, physical robot experiments showed that the first simplification matters: in real-time runs, a fixed-size filter that is too large for the processor drops readings and ends up less accurate than a smaller adaptive one (KLD-Sampling: 114 cm versus 44 cm average error). Modern simulation-only evaluations, such as differentiable filters in House3D (PF-net) or adaptive belief trees in POMDP benchmarks (AdaOPS), cannot see this effect at all.

We propose to make simulation honest about time and about model error, then test whether that honesty is enough to predict real-world results. The harness lets a filter's measured compute time advance the simulated clock and injects structured mismatch between the simulator and the filter's model. It is built as swappable modules, so any combination of filter model, particle-allocation policy, and clock can be run without rewriting code. The question is empirical and falsifiable: do rankings of allocation policies under time-honest simulation match rankings on paired real-world logs better than standard simulation does? A second, smaller contribution is an allocation rule that sizes the cloud by statistical need and also by the compute deadline.

Core Claim

Standard simulation evaluates adaptive particle filters with a bias whose direction we can predict. It favors large populations, because compute costs nothing. It also favors allocation rules that trust the current particle cloud, because the model is exactly right. Self Adaptive Particle Filter shows these are precisely the rules that underestimate the needed sample size when the proposal is mismatched. Here the "proposal" is the distribution the filter uses to generate new guesses, and it is mismatched when it differs from the true dynamics.

We claim that two changes recover the real-world ranking of allocation policies. The first is a compute-coupled clock, under which sensor frames that arrive while the filter is busy are lost. The second is mismatch injection, meaning correlated and biased noise, model perturbation, and latency. We claim neither change recovers the ranking alone. We also claim that a rule bounding the population by both statistical need and the per-frame budget beats either bound alone on real logs.

The existing work closest to this proposal falls into three groups, and none does what we propose. Fox observed the speed–accuracy U-shape on hardware but never built a simulator that reproduces it. SMCTC modularized particle filtering but fixed the population size; its author lists adaptive populations as future work, and it was evaluated only on synthetic problems. Habitat 2.0 adds a fixed one-step observation delay, which improved sim-to-real robustness, but that delay does not depend on how much the agent computes.

Research Program

The harness has three swappable pieces. The first is the model: the motion model and the sensor likelihood. We reuse SMCTC's separation of the generic filtering engine from these problem-specific "movesets." The second is the allocation policy, a module that at each step receives a summary of the particle cloud and the remaining time budget and returns a population size. The third is the clock. It comes in three versions: virtual time (the standard simulator), a fixed delay (as in Habitat 2.0), and compute coupling. Under compute coupling, the wall-clock time of each filter update, measured on the target hardware or a calibrated profile of it, decides which sensor frames the filter actually receives.

Letting the population size change at runtime breaks the static-shape contracts that modular interfaces usually assume. We will measure the overhead the abstraction adds against a hand-written loop, because a slow harness would itself distort the clock.

Mismatch injection is a fourth, separately configurable layer. It covers time-correlated and biased sensor noise, slow calibration drift, perturbed motion models, and occlusion dropouts. These are the structured failure modes the reality-gap literature attributes to i.i.d.-Gaussian simulators.

The allocation rule we test caps the statistical requirement with the time budget:

$$N_t = \min\left( N^{\text{stat}}_t,\ \left\lfloor \frac{T_{\text{frame}} - c_0}{c_p} \right\rfloor \right)$$

Here $N^{\text{stat}}_t$ is the KLD-sampling bound computed from occupied bins and corrected for proposal mismatch by Soto's relative numerical efficiency. $T_{\text{frame}}$ is the sensor interval, $c_0$ is the measured fixed cost per update, and $c_p$ is the measured cost per particle. When the effective sample size collapses, which signals model mismatch, the policy may knowingly exceed the cap and skip the next frame. Whether that trade is worth making is something the harness can measure.

The policy is deliberately simple. The scientific weight sits on the evaluation question, and the rule serves as one competitor among several.

The testbeds are chosen because they offer real data with a simulated counterpart. The first is 2D range-sensor localization in an office. We will collect real logs with laser-based ground truth, following Fox's Pioneer protocol, in a building we also model in simulation. The second is the Zhengzhou apple-orchard trajectories from YOLOv8-EK-IPF. These are 100 real RealSense sequences recorded at 5 Hz, and the paper reports results both on a simulated trajectory model and outdoors. That paper's own gaps already hint at simulation optimism: against an unscented Kalman filter, the simulated mean-error reduction was 29.50% but the outdoor reduction was 22.25%. The third is a high-dimensional stress test, NeRF localization on LLFF following Fast Loc-NeRF. There, rendering cost per particle dominates and bin-based bounds stop working. We replay those image sequences under the compute-coupled clock.

GLIMO appears to cut the other way, since it argues that imperfect simulators help transfer. We read it as compatible with this proposal and test that reading directly. If deliberate mismatch acts as a regularizer, then policies tuned under mismatch injection should transfer better than policies tuned in a clean simulator.

Baselines

The naive reference is a fixed population sized for global localization and run on the same hardware. This is the filter most simulation papers effectively evaluate, as in PF-net. We also include a fixed population tuned to the best single size, chosen by grid search under the same tuning budget given to every adaptive policy. That gives fixed-size filtering its strongest fair form.

The likelihood-scaled heuristic grows the population when the sensor likelihood drops. Fox reports it at 79 cm in real-time office runs, and we include it because it is the obvious cheap rule.

The strongest existing method is KLD-sampling (Fox), with 44 cm average real-time error on the Pioneer office benchmark. Our office logs are newly collected, so that figure is context rather than a target. We will re-run KLD-sampling on our logs with the same bin size, error threshold, and tuning budget as every other policy.

On the orchard data, the strongest competitor is the dynamic-allocation improved particle filter from YOLOv8-EK-IPF. It reports a 10.89% lower mean absolute error than an unscented particle filter outdoors. No absolute error value appears on Lacuna, so we will re-implement it and compare on identical sequences.

We also include Soto's entropy-triggered, relative-efficiency-corrected filter (Self Adaptive Particle Filter).

Two ablations isolate the proposal's ingredients. The first runs the deadline cap alone, without the relative-efficiency correction or the mismatch trigger. The second runs the statistical bound alone, without the cap.

On the evaluation side, the harness's compute-coupled clock is compared against the virtual clock and the fixed-delay clock. Each clock is crossed with mismatch injection on and off.

Two different outcomes count as a win. The allocation rule wins if it lowers real-log error relative to tuned KLD-sampling at equal or lower compute, with the advantage holding across seeds. The harness wins if it raises rank agreement with real logs above both the virtual and fixed-delay clocks.

Why not just…?

Why not just run a large fixed population on a GPU? For low-dimensional 2D localization this is a real threat, because cheap parallel particles may push the U-shape's turning point beyond any population anyone needs. The compute-coupled clock measures exactly this on the target hardware, and if the fixed-population filter is never starved of frames, we will report that adaptivity is unnecessary there. Where each particle needs rendering, the evidence points the other way: fixed-population Loc-NeRF took 0.724 s per update and diverged to 1.43 m error on "Orchids" (Fast Loc-NeRF). The NeRF testbed settles this case.

Why not just tune the population in a standard simulator and deploy? That is the practice under test. The decisive experiment is whether settings tuned under the virtual clock land near the real-log optimum. The orchard paper's shrinking advantage from simulation to outdoors suggests they will not, but that is one data point, so we measure it directly.

Why not just add a fixed sensor delay like Habitat 2.0? This is the most serious simple competitor. A fixed delay already improved transfer in Habitat 2.0, and it might capture most of the effect we attribute to compute coupling. Because a fixed delay does not change with population size, it cannot penalize large populations. Whether that difference matters is settled by the fixed-delay arm of the clock comparison. If the fixed-delay arm matches real rankings as well as compute coupling does, we will drop compute coupling and say so.

Why not just randomize noise heavily? Randomizing i.i.d. noise does not create correlated bias or proposal mismatch, and offline specialization transferred badly in targeted evolution of robot components. We include an i.i.d.-randomization arm alongside structured mismatch to test whether the structure matters.

Proposed Experiments

On each testbed we run every allocation policy under every clock-and-mismatch setting, with matched tuning budgets and ten seeds. We then run the same tuned policies on the real logs: the office logs and the orchard sequences.

The primary metric is Kendall rank correlation between the policy ranking in each simulator setting and the ranking on real logs. Secondary metrics are the gap between the error a simulator predicts and the error observed on real logs, localization error or trajectory mean absolute error, the fraction of frames dropped, and total compute.

The decisive check is pre-registered. Compute coupling plus mismatch must beat the virtual clock in rank correlation on both real testbeds. It must also beat the fixed-delay clock on at least one testbed, or the clock contribution is withdrawn.

A secondary check asks whether policies tuned under mismatch injection transfer better than clean-tuned policies, which tests the GLIMO reading. A final check is the measured overhead of the modular harness against a hand-written loop; if it exceeds a few percent of the frame budget, the clock is unreliable.

The main risk is cost. Collecting paired office logs with real ground truth, plus building a matching floor-plan simulator, is substantial work. If that collection fails, the orchard pair alone still tests the core claim, though with lower statistical power.