Many detection systems are not asked "what objects are here?" but "where is this object?" Examples include person search, one-shot detection from an example crop, and long-term tracking. The best systems for this question inject the example (the "query") into the very first stage, where candidate boxes are generated, so that a hard-to-see target is not thrown away before it can be matched. Deployed sensors also wear out. Elastomers fatigue, conductive paths crack, and gain and baseline drift, all slowly and unevenly across the sensing surface. We propose to treat this kind of progressive, internal sensor wear as a domain shift for query-guided detectors. We also argue that the query itself supplies an adaptation signal that ordinary class-agnostic detectors lack. Because the system knows what it is looking for, it can manufacture localization problems with exactly known answers from unlabeled worn data. It can then train on those problems instead of trusting a teacher network's guesses.
In two sentences: take a crop from an unlabeled frame recorded by the worn sensor, perturb it, and require the query-guided detector to find it again in that frame. The box is known by construction, so this supervises adaptation without the pseudo-labels that break down when degradation corrupts the proposal stage. The same re-finding task, run on held-out crops, gives a label-free score for choosing which adapted checkpoint to deploy.
Core Claim
Current unsupervised domain adaptation for detection increasingly relies on a slowly updated "teacher" copy of the detector whose soft predictions supervise a "student" on unlabeled target data. Align and Distill (ALDI++) unifies alignment and distillation. It reports this as the main source of recent gains, and it also shows that many older gains came from weak source baselines. That approach assumes the teacher's candidate boxes are roughly right. Under spatially uneven sensor wear, this assumption is weakest where it matters most: dead or drifting regions produce misplaced or missing proposals, and the student learns the teacher's mistakes. Our claim has three parts. First, query-guided detectors are more fragile than class-agnostic ones under wear, because early fusion correlates the query against distorted features and drops targets that cannot be recovered later. Second, a teacher-free "self-query" localization loss repairs this better than teacher distillation at equal compute. Third, self-query recall on held-out crops tracks true target accuracy well enough to replace the labeled target validation sets that Align and Distill criticizes current protocols for using.
We are not the first to combine queries and domain adaptation, so the contrast needs stating. AQT uses a learnable token as a query inside the discriminator, the network that tries to tell source from target features, so that it can hunt down domain-specific features in a transformer detector. Its object queries remain generic, and it studies only weather and synthetic-to-real shifts. MD-DETR uses localized proposal features as queries into frozen memory to fight forgetting when new classes arrive. It addresses shifts in which classes appear, not drift in how existing objects look. Domain Adaptive Faster R-CNN pushes the proposal network toward domain invariance, but that network stays blind to the query. None of these uses the target-conditioned localization task as the source of adaptation supervision, and none studies intrinsic sensor wear.
Research Program
A wear model grounded in durability physics. The sensor-durability literature describes the failure modes we need to simulate. Decoupling Transmission and Transduction shows that even a sensor surviving a million cycles without drift still has sensitivity that rises up to 25% with loading rate. It models this with a second-order viscoelastic model, which describes a soft material whose response depends on how fast it is loaded. ViTaM documents strain interference in stretchable tactile arrays: bending of the substrate produces noise that masks contact signals, and suppressing it raised force accuracy by 45.3%. Fatigue studies such as the fingerprint-inspired pressure sensor and the ternary nanocomposite sensor report endurance limits of tens of thousands to 100,000 cycles, and residual cross-axis error persists within those limits. GenForce shows that small hardness differences between nominally identical units already break learned models. These papers motivate the shift and constrain its form. The shift is fixed in sensor coordinates, it accumulates, and it is uneven across the surface.
We encode these properties as
$$x_t(u) = m_t(u)\,\big[g_t(u)\,(h_t * s)(u) + b_t(u)\big] + \epsilon,$$
where $s$ is the clean signal and $u$ indexes a sensor location. The wear age $t$ is measured in load cycles. $g_t$ is a smooth gain field that loses sensitivity at uneven rates, and $b_t$ is a drifting baseline offset. $h_t$ is a rate-dependent temporal lag kernel fit to the second-order viscoelastic behaviour, and it also carries cross-channel leakage. $m_t$ is a dead-channel mask that grows along simulated crack paths, and $\epsilon$ is noise. We apply this model to the images of CUHK-SYSU and PRW (person search) and to one-shot detection splits, producing a schedule of increasing wear levels. We hold out entire mechanism families, for example crack-growth masks or viscoelastic lag, for testing. We are explicit that this is a controlled stand-in. Cameras do not fatigue like elastomers. What we borrow is the structure of intrinsic wear: persistent, spatially fixed, progressive, and uneven.
The detector and the self-query loss. The base model is a query-guided two-stage detector in the style of QGN, which fuses query and scene features in a shared backbone and uses a query-guided proposal network to suppress non-matching people. We also run a co-attention variant following One-Shot Object Detection with Co-Attention and Co-Excitation. On unlabeled worn frames, we first sample a region $r$ that a class-agnostic proposal head rates as likely to contain an object. We then build a query from $r$ with strong photometric changes, rescaling, and extra channel dropout, so that pixel-exact matching is useless as a shortcut. Finally, we train the query-guided proposal and box heads to recover $r$ in the full worn frame, where other objects act as distractors. The loss is ordinary proposal classification plus box regression against a box known exactly.
Scene-level geometric transforms would move the wear pattern along with the content, which never happens physically. We therefore keep the scene untransformed so that wear stays fixed in sensor coordinates. Where video is available, as in PRW's multi-camera frames, we add forward-backward cycle consistency across frames, in the spirit of GlobalTrack-style search: content moves across the fixed wear, and a crop must return to its origin.
Label-free model selection. For every checkpoint along an adaptation run, we compute self-query recall on a held-out pool of worn crops that were never used in training. This is the fraction of those crops the detector re-finds among its top-$k$ proposals. We test whether this score ranks checkpoints the same way true target mAP does.
Baselines
The naive reference is the source-trained query-guided detector evaluated at each wear level with no adaptation. The same detector with a per-channel running normalization front-end serves as the "recalibrate the sensor" reference. It subtracts a running mean and divides by a running standard deviation at each sensor location, which is the flat-field correction a hardware engineer would apply first. Every other method is also reported with and without this front-end.
The strongest existing adaptation method is ALDI++ from Align and Distill. It combines a heavily augmented warm-up stage, a teacher updated as a moving average of the student, and soft distillation over classification, box, and objectness outputs. It is the right competitor because it is the method the file reports as closing to within 1.0 AP of a fully supervised target model on several benchmarks, with +3.5 target AP50 on Cityscapes to Foggy Cityscapes and +5.7 on Sim10k to Cityscapes. We run it on exactly the same unlabeled worn streams, with the same backbone, the same number of target images, the same GPU-hour budget, and the same tuning grid, and we select its hyperparameters with the same label-free criterion given to our method. We also report both methods under the conventional labeled-validation protocol. We additionally run AQT, whose reported 47.1 mAP on Foggy Cityscapes is the strongest transformer figure in the file, on a DETR-style query-guided variant.
For the query-guided side, the reference number is QGN's 91.5 mAP on clean CUHK-SYSU from Query-Guided Networks. For our worn variants there is no reported result found on Lacuna for any method, so all numbers under wear will be produced by us under one protocol.
The ablation that isolates the key ingredient uses the same query-guided detector with the self-query loss replaced by ALDI++-style teacher distillation, at matched compute. If this matches our method, the benefit comes from the architecture rather than from teacher-free supervision.
Beating the competition means three things at the heaviest held-out wear level, averaged over three seeds with non-overlapping confidence intervals. The first is higher person-search mAP than ALDI++. The second is higher recall of the queried target among the top 100 proposals. The third is a smaller gap to the clean-sensor score.
Why not just…?
Why not just recalibrate the sensor and run the unmodified detector? For pure gain and offset drift this may be nearly sufficient, and we treat that as a live possibility rather than a straw man. Dead channels and rate-dependent lag cannot be undone by per-location normalization. Even a sensor that shows no degradation over a million cycles retains a 25% rate-dependent sensitivity change (Decoupling Transmission and Transduction). The normalization baseline, reported per mechanism family, settles this. If it closes the gap on every family, the adaptation component is unnecessary and the benchmark result stands alone.
Why not just train with simulated wear as augmentation? This is the most serious threat, because we built the simulator and could simply train on it. The answer depends on unseen wear. We train the augmentation baseline on all mechanism families except the held-out one and test on the held-out family. If augmentation matches self-query adaptation there as well, our method's value reduces to convenience, and we will say so. The file supports some scepticism here, since cross-unit hardness differences alone break models in GenForce, but that finding does not substitute for the held-out test.
Why not just use a class-agnostic proposal network with more proposals? The person-search results in Neural Person Search Machines and Instance Guided Proposal Network indicate that inflating generic proposals floods the matcher with distractors. Wear may nonetheless hurt the class-agnostic stage less than query fusion does, so this baseline could win under heavy wear. We measure target recall and final mAP at 300, 1000, and 2000 proposals against the query-guided detector at each wear level. This comparison is also our first decisive check.
Why not just translate worn signals back to clean ones with a generative model? GenForce does this between fixed sensor designs with a diffusion model. It requires paired or well-characterized sensor states, whereas continuous wear gives a moving target with no clean reference. We include a CycleGAN-style unpaired translator trained on source versus current-wear frames as a baseline. We report its compute and accuracy, and specifically whether it hallucinates content in dead regions, which would mislead localization.
Decisive Experiments
The first experiment tests whether query fusion is a liability under wear, which is the tension the file identifies between QGN and ViTaM. We plot recall of the queried target against wear level for the query-guided and class-agnostic proposal stages. We then run the query-guided detector twice, once with fresh queries against a worn gallery and once with both worn, because this asymmetry is what deployment produces. A steeper decline for the query-guided stage would confirm the problem this proposal targets.
The second is the head-to-head described in Baselines, on CUHK-SYSU and PRW under progressive wear and on held-out mechanism families. We also run Cityscapes to Foggy Cityscapes with a one-shot formulation to check that the method does not hurt on ordinary weather shift.
The third tests label-free selection. Across all checkpoints of every run, we compute the Spearman rank correlation between self-query recall and true target mAP. We compare it with teacher-student agreement and prediction entropy, and we report the mAP lost by choosing the top label-free checkpoint instead of the labeled-best one. Two outcomes would each count against the proposal: a correlation not clearly above the alternatives, or self-query adaptation failing to beat the teacher-distillation ablation at matched compute.
The fourth is a limited real-signal check. We use tactile pad streams from DECO-50 and derive contact-region boxes by thresholding clean frames. We then query by an earlier contact footprint under applied wear, testing whether the conclusions carry over from camera images to a spatial sensor of the kind that actually fatigues. Real cyclic wear recorded on hardware remains outside what the file lets us ground, and we flag it as the next step.
References
- Align and Distill: Unifying and Improving Domain Adaptive Object Detection
- AQT: Adversarial Query Transformers for Domain Adaptive Object Detection
- Preventing Catastrophic Forgetting Through Memory Networks in Continuous Detection
- Domain Adaptive Faster R-CNN for Object Detection in the Wild
- Decoupling Transmission and Transduction for Improved Durability of Highly Stretchable, Soft Strain Sensing: Applications in Human Health Monitoring
- Capturing forceful interaction with deformable objects using a deep learning-powered stretchable tactile array
- Deep Learning‐Assisted Fingerprint‐Inspired Flexible Pressure Sensor for Tension Monitoring in Carbon Fiber Production
- Stretchable, Highly Durable Ternary Nanocomposite Strain Sensor for Structural Health Monitoring of Flexible Aircraft
- Training tactile sensors to learn force sensing from each other
- Query-guided networks for few-shot fine-grained classification and person search
- One-Shot Object Detection with Co-Attention and Co-Excitation
- GlobalTrack: A Simple and Strong Baseline for Long-Term Tracking
- Neural Person Search Machines
- Instance Guided Proposal Network for Person Search