Telling Target Loss from Wire Loss: Hardware-Aware Re-Detection Triggers for Analog Trackers

Long-term visual trackers save compute by running a cheap local tracker on every frame. They wake an expensive global detector only when the tracker's own confidence score drops. Analog in-memory computing is a natural home for that always-on local loop. These are chips that do matrix multiplication inside resistive memory arrays, called crossbars, using Ohm's and Kirchhoff's laws. But the metal wires in a crossbar have resistance. As current flows along them the voltage sags (the "IR-drop"), and every cell far from the driver computes with a weakened signal. The sag depends on how much total current the frame's activations draw. A bright, cluttered frame therefore distorts the network more than a dark, plain one, even when the target is perfectly visible. A confidence score computed on such hardware can fall for a reason that has nothing to do with the target. The re-detection gate would then fire on wire loss rather than target loss. That wastes energy, and it can also hide real failures.

This note proposes to measure that confusion and remove it with a signal the chip already has. In a crossbar, the digital-to-analog converters know each frame's input magnitudes, and the analog-to-digital converters read out the column currents. Summing those readouts gives a per-frame, per-layer "electrical stress" estimate that predicts how hard IR-drop is hitting at that moment, at almost no added cost. The core claim is testable. Conditioning the re-detection decision jointly on visual confidence and measured electrical stress should separate hardware-induced confidence drops from real tracking failures. That should recover long-term tracking accuracy at a fixed detector budget, beyond what a one-time threshold recalibration achieves.

Core Claim and What Is New

Two bodies of work never meet. Crossbar papers model IR-drop carefully but only on static single-image classification. Energy-Accuracy Trade-Offs for Resistive In-Memory Computing Architectures builds a behavioral model in which wire parasitics cap compute accuracy at an 18–22 dB signal-to-noise-plus-distortion ratio. It also shows that high-conductance devices such as MRAM lose about 11 dB because they draw more current. GENIEx trains a small network to emulate HSPICE-level crossbar distortion. The two Bhattacharjee et al. studies (batchnorm limits; Examining the Role and Limits of Batchnorm Optimization to Mitigate Diverse Hardware-noise in In-memory Computing, sparse DNNs) propose batchnorm fine-tuning, column rearrangement and weight-constrained training. They also show that linear correction stops working once distortion becomes strongly nonlinear.

All four papers treat distortion as a uniform degradation of classification accuracy. None asks what it does to a decision variable that gates computation over time.

Tracking papers do the opposite. UAST predicts distributions over box edges plus a joint classification-localization confidence. RTracker replaces absolute thresholds with a relative test against a tree of past target appearances and stored distractors. STORM uses an energy-like compatibility score with statically calibrated thresholds. All of them assume the score is computed exactly on a GPU.

The closest hardware trackers also sidestep the issue. Analog VLSI Circuits for Attention-Based, Visual Tracking is a 23-pixel focal-plane chip with a hardwired winner-take-all. Event-based attention and tracking on neuromorphic hardware runs on fully digital Loihi. Neither has resistive arrays, IR-drop, or a confidence-gated detector.

The new element here is specific. It is a per-frame calibration of the re-detection trigger that is conditioned on electrical stress the crossbar reports about itself. This rests on the empirical claim that stress explains a meaningful share of confidence variance on visible-target frames. If that claim fails, the proposal says so and reduces to a simpler, still useful result: a fixed threshold shift is all that is needed.

Research Program

Simulation substrate. The local tracker is a Siamese tracker in the UAST style. Its backbone and correlation layers are mapped onto crossbar tiles of 32×32, 64×64 and 128×128. Each tile's output is computed by solving the resistive network with nonzero wire resistance, which is exact for linear devices. A GENIEx-style learned emulator, checked against the solver on sampled tiles, provides speed on full benchmarks. Device parameters span low-conductance ReRAM/FeFET-like and high-conductance MRAM-like settings, following Roy & Shanbhag. aihwkit supplies read noise and conductance variation layered on top.

The global re-detector and any memory verifier stay digital. This isolates the trigger as the variable under study and matches the deployment picture in which the heavy module is power-gated.

Diagnosis. Every benchmark frame is run twice, once through the ideal network and once through the IR-drop network. This gives a direct decomposition of each confidence change into a visual part and a hardware part. On frames where the target is annotated as visible and the ideal tracker is on target, we regress the hardware-induced confidence change on the stress features. Those features are the summed column outputs and input norms per mapped layer. This gives a falsifiable number: the fraction of hardware-induced confidence variance that stress explains beyond a constant offset. Following the batchnorm-limits analysis, we also track their second-principal-component nonlinearity metric per frame. That shows whether video scenes push layers past the regime where linear correction holds.

Mechanism. The calibrated trigger estimates

$$p(\text{on target} \mid c, s),$$

where $c$ is the raw tracker confidence (UAST's joint score) and $s$ is the low-dimensional stress vector for the current frame. It is fit with a small logistic or binned-isotonic model on training sequences run through the simulator. Re-detection fires when this probability falls below a threshold. The threshold is chosen to hit a target detector invocation rate.

The probe is deliberately tiny. The proposal should not succeed merely by adding a learned model. A second variant applies the same conditioning to RTracker's relative distance test. This asks whether a relative test is already immune: because RTracker compares against its own past appearances, it may inherit the same IR-drop distortion and partly cancel it.

Predictions. The physics makes predictions beyond "it helps." The benefit of stress conditioning should grow with array size and with device on-conductance, because both raise total current and hence IR-drop. The benefit should vanish in the zero-wire-resistance limit. A result that ignores these trends would point to the probe exploiting something other than parasitics.

Baselines

The digital reference is the same local tracker, trigger and detector run in floating point. It is the ceiling the analog system tries to approach. The file reports no long-term-wrapper numbers for it, but UAST alone reports 0.571 LaSOT success and 0.635 GOT-10k Average Overlap (AO) in UAST.

The naive analog reference runs the tracker on the simulated crossbar with the threshold calibrated on GPU scores. It represents doing nothing about the hardware.

The strongest simple competitor is static recalibration. It uses the same analog tracker, but re-tunes the scalar threshold, or a monotone remapping of $c$ alone, on the same simulated training sequences used to fit our probe.

A hardware-aware training baseline fine-tunes the tracker with crossbar noise and IR-drop emulation in the loop using aihwkit-style training. Its threshold is then statically recalibrated. We also apply batchnorm fine-tuning per Bhattacharjee et al. on the same data.

An input-normalization baseline rescales each search window's input vector to fixed energy before the digital-to-analog converters. This directly attacks the data dependence.

The strongest existing long-term method is RTracker. It reports 74.7% AUC on LaSOT and 63.2% AUC on VideoCube on digital hardware. We run it both digitally and with its local backbone on the simulated crossbar. On analog hardware, no reported result was found on Lacuna for RTracker or any tracker.

All methods share training videos, simulator settings, and calibration-set size. Every trigger is compared at matched detector invocation rates, swept as a curve, so no method wins by simply firing the detector more.

Two ablations isolate the key ingredient. In one, the stress vector is replaced with stress from a randomly chosen other frame of the same sequence, which keeps marginal statistics but breaks the frame-level link. In the other, stress is used without visual confidence.

"Beating" static recalibration means a statistically clear AUC gain on LaSOT and VideoCube at matched invocation rate. That gain should close a substantial part of the naive-to-digital gap. It should also lower the false trigger rate on visible-target frames, and it should grow with array size and conductance as predicted.

Why not just…?

Why not just recalibrate the threshold once on analog-simulated scores? This is the real threat. If IR-drop mostly scales all confidences by a near-constant factor, one offline shift captures it. Our diagnosis experiment measures exactly this. If stress explains little variance beyond a constant offset, static recalibration wins and we report that result. Our expectation that it falls short rests on the strict data dependence of IR-drop emphasized by Roy & Shanbhag and GENIEx. Long videos in LaSOT swing between lighting and clutter conditions within a single sequence, so a fixed offset is unlikely to fit the whole sequence.

Why not just train the tracker to be hardware-aware? Noise-injection training makes features robust on average. But it cannot know at test time which frame is electrically stressed. Bhattacharjee et al. show linear compensation fails once nonlinear distortion passes a threshold. Their sparse-DNN study still finds about a 21% accuracy drop for VGG11 on 64×64 arrays without further mitigation. The two approaches are complementary, so we test the probe on top of the hardware-aware model as well. A gain there is the stronger evidence.

Why not just normalize the input energy per frame? This fixes the row-side drive. But column currents still depend on which weights the active inputs hit, so IR-drop stays content-dependent. Normalization also discards contrast information the tracker may use. Running it as a baseline settles empirically how much data dependence survives.

Why not just run the confidence head digitally? The head is small, but its inputs are the distorted analog features. Moving it to digital does not remove the distortion. The digital-head setting is in fact our default configuration for the final scoring layer, so any remaining gap is attributable to upstream IR-drop.

Proposed Experiments

The main evaluation uses LaSOT and VideoCube. Both are long, have frequent out-of-view and occlusion events, and have reported RTracker numbers. GOT-10k AO and VOT-2019 Expected Average Overlap serve as short-term checks that calibration does not hurt steady-state accuracy. UAST reports 0.334 Expected Average Overlap on VOT-2019.

Metrics are success AUC and precision. Trigger quality is scored against ground-truth target-absent and full-occlusion frames, as precision and recall of re-detection firings. The false trigger rate is measured on visible, correctly tracked frames. Detector invocation rate serves as the energy proxy.

The first decisive check is the variance decomposition on visible-target frames. If stress beyond a constant explains only a small fraction, the conditioning idea is dropped in favor of static recalibration. The second is the matched-invocation-rate comparison against static recalibration and hardware-aware training. The third is the physics sweep over array size, wire resistance and on-conductance. Here the probe's advantage should rise with array size and conductance and disappear at zero wire resistance. The shuffled-stress ablation must lose most of the gain.

A final stress test is a controlled perturbation set. It applies brightness and clutter changes to LaSOT frames where the target remains visible. This directly counts how often each trigger fires on wire loss rather than target loss.