Threshold-Fired Global Search for Tracker Recovery

A visual tracker follows one object through a video by looking only in a small window around where the object was last seen. When the object is hidden or leaves the frame, the tracker keeps looking in the wrong place. It locks onto background or a look-alike object and then corrupts its own memory of the target. The known fix is to search the whole frame for the target, but full-frame search is slow: GlobalTrack runs at 6 frames per second and the full Siam R-CNN at 4.7 on a V100 GPU. So the practical question is when to pay for a full-frame search.

We propose to answer that with a timing rule borrowed from a laser. In a self-optical parametric oscillator, a second wave (the "idler") gets no feedback from the cavity mirrors. It builds up only when the pump pulse passes a gain threshold near its peak. As a result it releases its energy in a burst about four times shorter than the pump: 4.78 ns against 21.9 ns in A 100 Hz, 4.71 mJ, 3.8 μm ... self-optical parametric oscillator. We use this as a design lens, not as hardware. Whole-frame search should behave like that idler. It is driven by an accumulated signal that the target is lost, stays off below a threshold, fires as a short burst when the threshold is crossed, and is shut off by its own success, the way the idler stops once it drains the pump.

Core Claim

Recovering from occlusion is a scheduling problem, and current trackers schedule badly in one of two ways. Some search the whole frame on every frame: GlobalTrack, which uses no memory between frames (GlobalTrack), and Unicorn, which matches the full reference frame against the full search frame (Towards Grand Unification of Object Tracking). These waste compute while the target is plainly visible. Others react to a single frame of low confidence, which fires too often when confidence is noisy and too late when the tracker drifts slowly. RTracker improves on fixed thresholds with a memory tree of target and distractor appearances that pauses tracking when the target vanishes (RTracker), but it still costs about 0.75 times the base tracker's speed.

Our claim is that a trigger with integration, two thresholds, and a reset on success can match always-on global search on recovery while running the expensive search on a small fraction of frames.

The trigger keeps a running "lost" signal

$$e_t = \lambda e_{t-1} + (1-\lambda)(1 - c_t),$$

where $c_t \in [0,1]$ is the local tracker's confidence at frame $t$ and $\lambda \in [0,1)$ sets how much past evidence carries forward. Two thresholds act on $e_t$. Above the lower one, $\theta_{\text{freeze}}$, the tracker stops updating its stored appearance of the target. Above the higher one, $\theta_{\text{fire}}$, the full-frame detector runs. A confident global detection resets $e_t$ to zero.

The low freeze threshold is the second half of the idea. Once a tracker stores a false detection as the target's appearance, that damage cannot be undone (Re2EMA). Freezing updates early, before committing compute to a search, should prevent most of that damage.

Research Program

We will build a hybrid from parts that already exist. A fast local Siamese tracker provides $c_t$. The GlobalTrack query-guided detector serves as the full-frame search. The trigger sits between them.

While $e_t$ stays above $\theta_{\text{fire}}$, which corresponds to a long disappearance, the detector runs on a fixed duty cycle. We will report how recovery latency depends on that duty cycle, rather than hiding it.

The analysis has three parts. First, we will trace a compute-versus-recovery curve by sweeping $\lambda$ and the thresholds. Second, for every annotated disappearance we will measure recovery latency: the number of frames between the target reappearing and the tracker correctly re-acquiring it. Third, we will measure how often the stored target appearance absorbs a false detection, which we can count wherever the benchmark provides ground-truth boxes.

Baselines

The do-nothing reference is the local Siamese tracker with no global search at all. Performance Evaluation Methodology for Long-Term Single-Object Tracking reports SiamFC at a Tracking F-score of 0.37 on LTB50. In that paper's artificial jump experiment, where the target is moved to a different quadrant of the frame, SiamFC and MDNet fail. This baseline marks the floor.

The compute ceiling is always-on GlobalTrack, with search on every frame. It reports 52.1% AUC on LaSOT and 63.8% on TLP at 6 frames per second (GlobalTrack). Our hybrid uses the same detector weights, so any accuracy gap comes from scheduling alone. We beat it if we come within one AUC point while invoking the detector on under 20% of frames.

The strongest accuracy competitor is RTracker, which reports 74.7% AUC on LaSOT, 63.2% on VideoCube, and re-detection within 2–5 frames of reappearance (RTracker). Its base tracker is stronger than ours. For a fair comparison we will also attach our trigger to RTracker's own base tracker and its own global detector, with the same training data and the same threshold-tuning budget, tuned on LaSOT training sequences only. Beating it means equal or better recovery latency and AUC at higher frames per second.

On LTB50, the reference is FCLT at a Tracking F-score of 0.41. It is also the only tracker in that study that re-detected the target in all 50 jump sequences (Performance Evaluation Methodology).

The key ablation replaces the integrated trigger with an instantaneous threshold on $c_t$, including a variant that requires $k$ consecutive low-confidence frames before firing. A second ablation merges the two thresholds into one ($\theta_{\text{freeze}} = \theta_{\text{fire}}$), which isolates the effect of freezing template updates early.

Why not just…?

Why not just use a tuned confidence threshold with a k-frame debounce? This is the real threat. A debounce is a crude integrator, and if confidence noise is small it may perform the same as our trigger. The key ablation settles the question directly. We expect a difference only where confidence decays slowly during drift onto a distractor, which can be detected only by integrating over time. If the two curves coincide, we will report the compute-versus-recovery curves and the corruption measurements as the contribution, and drop the claim that the trigger itself is better.

Why not just run global detection every frame, since GPUs get faster? At equal hardware, GlobalTrack's 6 frames per second against our trigger's mixed workload is a ratio set by how often the detector runs. That ratio does not shrink when hardware improves. LTB50 sequences average 10.5 disappearances lasting about 52 frames, so the target is visible on most frames and searching the whole image on those frames is wasted work.

Why not use the mid-infrared pulsed laser itself to see through occluders? No dataset in reach has synchronized active depth-gated returns with tracking annotations. No reported result was found on Lacuna for active-illumination occlusion recovery. That would be a hardware program, not a testable tracking claim, so the optics enters only as the source of the threshold-burst design.

Proposed Experiments

We will run on LTB50, with its 525 disappearance events and its quadrant re-detection test; on LaSOT, with 1,400 sequences of about 2,500 frames each (LaSOT); and on VideoCube, using its out-of-view subset. The metrics are Tracking F-score, AUC, recovery latency per disappearance, fraction of frames on which the detector runs, frames per second, and the rate at which the stored target appearance is contaminated by false detections.

Three checks decide the outcome. The first is the ablation against the debounced threshold on the LTB50 disappearance subset. The second is whether early freezing measurably lowers contamination and the error that follows it. The third is whether the hybrid passes the jump test in all 50 sequences, as FCLT does, while calling the detector rarely.