Person search means finding the same pedestrian across raw, uncropped frames from many non-overlapping cameras. It requires two skills: drawing a box around each person (detection) and recognizing who that person is across viewpoints (re-identification). Recent "weakly supervised" person search systems remove the expensive identity labels. They still assume a human has drawn a tight box around every pedestrian in every training frame, so the manual bottleneck has moved rather than disappeared. Region Siamese Networks names training without any box annotations as an open frontier (Weakly Supervised Person Search with Region Siamese Networks).
We propose to train person search on a new camera network with no boxes and no identities from that network. The core idea is that the identity-matching task can grade its own boxes. A box that cuts off a pedestrian's legs, or swallows a slab of camera-specific background, produces body-part features that disagree with the whole-crop feature about who the person's look-alikes are in other cameras. We turn that disagreement into a box-quality score. That score then selects and refines pseudo-boxes from a cheap, imprecise source detector. In return, only well-localized boxes are allowed to shape the identity clusters. This breaks the loop in which bad boxes corrupt identity pseudo-labels and noisy identity labels cannot correct bad boxes.
Core Claim
The obvious way to remove target-domain boxes is to run an off-the-shelf person detector and feed its output to an existing weakly supervised person search method. Nobody in the material reports how well this works. It is also the baseline we must beat, not a strawman.
Our claim is narrower and testable. The gap between that detector-fed pipeline and the same method trained on human boxes comes mostly from boxes that are slightly wrong, with overlap between roughly 0.5 and 0.7, rather than from missed people. A box-quality score derived from cross-camera, part-versus-whole agreement recovers a large share of that gap. Neither the detector's confidence nor a generic box-regression module can do this. The motivating evidence is that identification degrades sharply unless detections overlap the true person by more than 0.7 IoU (IoU is intersection-over-union, the standard box-overlap ratio) (Person Re-identification in the Wild). Detector confidence is unreliable exactly near the foreground/background decision boundary (Combating Noise: Semi-supervised Learning by Region Uncertainty Quantification).
The three directions play different roles. The weakly supervised localization gap is the mechanism question: what weak signal can tell a full-body box from a partial one when no box labels exist? The annotation bottleneck is the application regime, a new camera network with zero manual labels of any kind. Pseudo-label noise is the constraint. Both boxes and identities are model-generated, and their errors compound.
Mechanism
Part-based Pseudo Label Refinement (Part-based Pseudo Label Refinement for Unsupervised Person Re-identification) computes a Cross Agreement Score: the overlap between the nearest neighbors of a crop's global feature and those of each horizontal body stripe. It uses that score to clean identity labels on hand-cropped images. The research file notes its weakness: under misaligned crops the stripes no longer correspond to anatomy, and the score collapses. We invert that weakness into a feature. If stripe agreement collapses when the box is wrong, agreement measures box correctness.
For a detected person $i$ we generate candidate boxes $b \in B_i$. These come from jittered and rescaled versions of the source detection plus class-agnostic attention regions from a self-supervised vision transformer. DINO's attention tends to cover whole objects rather than single discriminative parts, as DiPS showed (DiPS: Discriminative pseudo-label sampling with self-supervised transformers for weakly supervised object localization). We score each candidate as
$$s(b) = \frac{1}{P}\sum_{p=1}^{P} \mathrm{J}\big(N_k^{\neg c}(g_b),\, N_k^{\neg c}(f_{b,p})\big),$$
where $g_b$ is the global embedding of the crop, $f_{b,p}$ is the embedding of its $p$-th of $P$ horizontal parts, $N_k^{\neg c}(\cdot)$ is the set of $k$ nearest neighbors in a memory bank restricted to cameras other than the crop's own camera $c$, and $\mathrm{J}$ is Jaccard overlap (shared neighbors divided by total distinct neighbors).
The restriction to other cameras matters. It borrows the insight of camera-aware proxies (Camera-aware Proxies for Unsupervised Person Re-Identification) that unsupervised features cluster by camera background. A background-heavy box finds its neighbors through scenery. Once same-camera neighbors are excluded, its background stripes find no consistent partners, and the score drops. A box truncated at the waist has leg stripes that match arbitrary lower-body textures, which again lowers agreement.
Training then alternates in the style of the expectation-maximization loop used by bounding box adjusters (Boosting Weakly Supervised Object Detection via Learning Bounding Box Adjusters). In the first step, each person's pseudo-box is sampled from its top-scoring candidates with probability proportional to $s(b)$. This follows DiPS's stochastic top-K sampling, but our open-set agreement score replaces DiPS's closed-set classifier, which cannot exist when the "classes" are unknown individuals. In the second step, a one-step person search network is updated on the sampled boxes. The box-regression loss and the identity contrastive loss are both weighted by $s(b)$. Identity pseudo-labels come from clustering only the instances whose best score exceeds a running quantile.
The network's own box head then proposes new candidates for the next round. Over time this lets localization improve beyond the source detector's systematic biases. Without that, the method would hit the "teacher ceiling" the file describes, where a student inherits its teacher's blind spots.
Research Program
The work has three parts.
The first is a measurement study, which we consider valuable even if the method fails. We train the strongest existing weakly supervised person search method on PRW using detector-generated boxes instead of human boxes. We then decompose the resulting loss by error type: missed people, false positives, and boxes overlapping the truth at 0.5–0.7 versus above 0.7. PRW's human boxes are used only to audit, never to train. This tells us whether the gap really lives in the slightly-wrong-box band our method targets.
The second part validates the score itself. Before any training loop, we check whether $s(b)$ ranks candidate boxes by true IoU better than detector confidence does, when computed on a frozen backbone. We also check whether the cross-camera restriction is responsible for penalizing background.
The third part is the full alternating training on PRW and CUHK-SYSU, with ablations that remove each ingredient. A cautionary note on setting: pure image-level tags carry almost no information in surveillance frames, because nearly every frame contains people. The weak localization prior therefore comes from a source detector trained on generic person boxes (COCO person or CUHK-SYSU boxes with identities discarded) and from self-supervised attention, never from target-network annotation.
Baselines
The naive reference point is direct transfer. A Faster R-CNN person detector trained on COCO persons (and, separately, on CUHK-SYSU boxes without identities) is run on PRW. Its detections above a fixed confidence threshold are fed unchanged into the person search learner. This is what a practitioner would do first. It tests whether any of our machinery is needed. Domain shift is known to hurt such transfer; task-sensitive alignment alone gave nearly 10% mAP in Domain Adaptive Person Search. However, no reported result was found on Lacuna for a weakly supervised person search learner trained on transferred pseudo-boxes, so we will establish this number ourselves.
The strongest existing method is the multi-granularity graph approach (Multi-Granularity Graph-Convolution-Based Method for Weakly Supervised Person Search). It aligns detection and re-identification features at cluster level, learns invariance to box drift, and refines identity labels with graph convolutions. It reports 37.6% mAP on PRW and 88.4% mAP / 89.9% top-1 on CUHK-SYSU, but with full human boxes. We use it in two ways. With human boxes, it is the upper bound we try to approach. With detector pseudo-boxes, it is the competitor we must beat. Its box-drift invariance is the existing answer to "boxes are imperfect," so it is the most dangerous rival.
We also run Deep Intra-Image Contrastive Learning (Deep Intra-Image Contrastive Learning for Weakly Supervised One-Step Person Search) on the same pseudo-boxes. It reports 35.5% mAP on PRW with human boxes. Its spatial-invariant contrast matches many drifted proposals to one crop, and removing that component cost 7.6% mAP. It is therefore the natural "just tolerate drift" competitor.
For context, the domain-adaptive one-step method FOUS (Fast One-Stage Unsupervised Domain Adaptive Person Search) reports 35.4% mAP on PRW. It needs a source domain with boxes and identities, so it uses strictly more supervision than we do.
All methods share our backbone, our source detector, the same number of training epochs, and the same hyperparameter search budget, tuned on a held-out split of PRW training cameras. Beating the competitor means two things. First, higher PRW mAP than the multi-granularity method trained on identical pseudo-boxes, by a margin exceeding three-seed variance. Second, recovering at least half of the gap between that pseudo-box result and its 37.6% human-box result.
The key ablations isolate the ingredient. We replace $s(b)$ with detector confidence, with the agreement score computed over all cameras rather than other cameras only, and with uniform (unweighted) sampling over the same candidate pool. We also drop the self-supervised attention candidates.
Why not just…?
Why not just use a COCO person detector and existing weakly supervised person search? This is the real threat, and the proposal is built around it rather than around beating weaker straw baselines. If the measurement study shows the transfer gap is dominated by missed people, our method cannot help. We would then report that finding, and the contribution shrinks to the diagnosis. The 0.7-IoU sensitivity reported in Person Re-identification in the Wild is why we expect slightly-wrong boxes to matter more than misses, but only the decomposition settles it.
Why not just trust drift-tolerant losses like spatial-invariant contrast? Those losses make features robust to boxes that wobble around a correct anchor. They never change which box is used for detector training or cluster formation. A systematically truncated box is consistently wrong, and tolerance to wobble does not fix it. The test is direct: if DICL or the multi-granularity method on raw pseudo-boxes matches our method, the idea is redundant and we will say so.
Why not just filter detections by confidence, or refine them with a generic box adjuster? Confidence is least reliable in the 0.5-overlap band, as shown in Combating Noise: Semi-supervised Learning by Region Uncertainty Quantification. A class-agnostic adjuster trained on COCO, like LBBA, learns what generic person boxes look like, not what this camera network's pedestrians need. We include both as ablation arms with matched candidate pools. If the adjuster alone matches the agreement score, the identity feedback is unnecessary, and we would drop the claim that identity matching supplies localization signal.
Proposed Experiments
The decisive first check comes before any training loop. On PRW, using frozen features, we measure the rank correlation between $s(b)$ and true IoU over candidate boxes, alongside detector confidence. If agreement does not rank boxes better than confidence, especially in the 0.5–0.7 band, the project stops there.
The second check is the full comparison on PRW and CUHK-SYSU. We report re-identification mAP and top-1 together with detection average precision and recall at IoU 0.5 and 0.7, measured against held-out human boxes. This shows whether retrieval gains come from better localization rather than better clustering alone.
The third check tests the closed loop for failure. We plot pseudo-box IoU and cluster purity per round. These must improve together. If they drift apart, the loop has slipped into confirmation bias.
Finally, a cross-network test trains the source detector on CUHK-SYSU street and movie imagery and adapts to PRW's fixed campus cameras. Because the agreement score depends on having multiple cameras, we also report how gains scale with the number of cameras in the memory bank. This distinguishes a genuinely cross-camera signal from generic self-consistency.
References
- Weakly Supervised Person Search with Region Siamese Networks
- Person Re-identification in the Wild
- Combating Noise: Semi-supervised Learning by Region Uncertainty Quantification
- Part-based Pseudo Label Refinement for Unsupervised Person Re-identification
- DiPS: Discriminative pseudo-label sampling with self-supervised transformers for weakly supervised object localization
- Camera-Aware Proxies for Unsupervised Person Re-Identification
- Boosting Weakly Supervised Object Detection via Learning Bounding Box Adjusters
- Domain Adaptive Person Search
- Multi-Granularity Graph-Convolution-Based Method for Weakly Supervised Person Search
- Deep intra-image contrastive learning for weakly supervised one-step person search
- Fast One-Stage Unsupervised Domain Adaptive Person Search