Person re-identification means finding the same pedestrian across different surveillance cameras. It fails most often when part of the person is hidden behind a car, a pole, or another pedestrian. The model then compares a visible torso in one image against hidden legs in another, or mistakes the occluder for part of the body. Current fixes rely on external pose detectors, which break under exactly these occlusions. Others prune image regions by attention, which can throw away small identity cues, or reconstruct missing parts from gallery images that may not exist. We propose a different route. Take the transformer's own attention map over image patches, then summarize its shape with persistent homology, a tool that tracks how connected blobs appear and merge as a threshold is swept. The threshold is mixed with each patch's vertical position, so the summary records not just how many body pieces there are but at what height each one sits.
The key observation is that pedestrians come with a natural axis, running from head to feet. "Fused filtration", which adds a spatial coordinate to the signal before computing topology, has so far needed a randomly rotated direction every training step, and its authors name this as a weakness. For upright people the vertical axis is the canonical choice, so no rotation is needed. The result is a compact, keypoint-free, differentiable description of which vertical extent of the target body is visible and connected. The model uses it in two ways. During training, a loss teaches attention not to fragment or spill onto occluders. At test time, the description decides which horizontal body stripes are compared between two images. On a 16×8 patch grid the computation is a union-find pass over 128 nodes, so the usual cost objection to topological methods does not apply.
Core Claim
A height-fused persistence diagram of a vision transformer's patch attention gives a reliable, detector-free estimate of target-body visibility. Training attention to make this diagram stable under synthetic occlusion, and using it to restrict matching to jointly visible body regions, will improve retrieval on real-occluder benchmarks. It will also generalize from synthetic to real occluders better than two cheaper alternatives: a learned visibility head and row-projection heuristics. The claim is falsifiable. If a simple threshold-plus-connected-components visibility estimate, or a supervised stripe-visibility head, matches it, then the topological machinery is not earning its place.
Background and Positioning
The method comes from Oner et al. In Localized Persistent Homologies for more Effective Deep Learning and Persistent Homology With Improved Locality Information for More Effective Delineation, they filter $Y[p] + \rho(p)$ rather than $Y[p]$ alone. Here $Y[p]$ is the pixel value at position $p$ and $\rho$ is a directional ramp. Identical features at different positions are therefore born at different thresholds. They show the resulting loss is close to monotone in injected errors, whereas standard persistence losses decreased about 40% of the time because errors were matched to distant, unrelated features. We reuse this locality property directly, but on attention maps rather than segmentation masks, and for metric learning rather than delineation. Their road and neuron experiments involve neither non-rigid humans nor retrieval. Curry's The Fiber of the Persistence Map for Functions on the Interval explains why the spatial coordinate must be fused in at all: the persistence diagram is exactly the summary that forgets where features sit. That is fatal for a visibility estimate.
TopoImages is the closest attempt to give a classifier localized topology. It computes persistence per fixed square patch, offline, on raw intensities, for medical classification, and lifts ResNet-50 AUC on ISIC 2017 from 79.01% to 81.47%. We differ on three axes. Our persistence is computed on learned attention rather than pixels. It is computed online and is differentiable. Its locality comes from a fused coordinate, not a rigid grid that cuts through limbs. We deliberately do not compute topology on raw pedestrian pixels, because clothing texture topology carries little identity signal compared with foundation-model features.
On the re-identification side, three papers motivate the gap and two constrain the design.
HOReID builds a 14-keypoint graph from an external pose estimator and propagates features from visible to occluded joints. Its "topology" is a kinematic graph whose quality is capped by the detector. Our visibility signal comes from the model's own attention.
FPC shows that transformer attention already separates occluders from pedestrians. It prunes low-attention tokens and reconstructs missing query patches from gallery neighbours. Its own ablation, in which dropping salient tokens causes collapse, warns us not to discard tokens. Its reliance on gallery consolidation is the dependency we remove.
ProFD uses CLIP text prompts as part priors and still needs noisy segmentation masks to ground them spatially.
DRFormer handles the local-global trade-off with learned bidirectional cross-attention between DINO and CLIP, plus two hand-designed regularizers. That work is the strongest evidence that local and global signals need an explicit mediator. Our mediator is structural rather than learned. A single global diagram encodes local positions, and it decides which local stripe features the global distance uses.
Research Program
Visibility descriptor. The backbone is a ViT-B/16 on 256×128 crops, the backbone FPC uses. From the last layer we take the class-token attention over patches, giving a map $a(p) \in [0,1]$ on the patch grid. We compute 0-dimensional persistence of the superlevel filtration of
$$g(p) = a(p) - \lambda\, h(p),$$
where $h(p) \in [0,1]$ is the normalized row index (0 at the top) and $\lambda \geq 0$ sets how strongly height staggers birth times. With $\lambda = 0$ this is ordinary persistence. With $\lambda > 0$, foreground blobs at different heights are born at different values. A merge then records that two height bands are joined through high-attention patches.
From the diagram we identify the dominant component, the one containing the highest-attention token. We read off its vertical extent and any long-lived secondary components. A secondary component is either a detached body part separated by an occluder, such as a pole crossing the thighs, or a distractor such as a neighbouring pedestrian. A width-and-overlap test against the dominant component's column span separates the two cases. As an optional extension, 1-dimensional features capture occluders enclosed inside the body silhouette. The descriptor is converted to a soft visibility weight $v_k \in [0,1]$ for each of $K=6$ horizontal stripes. Gradients flow through the critical patches, as in Oner et al.
Topological consistency loss. During training we paste synthetic occluders onto each image: random crops from other images, poles, and pedestrian cut-outs. Let $D(\cdot)$ denote the height-fused diagram, and let $M$ be the occluder mask. We penalize
$$\mathcal{L}_{\text{topo}} = W\big(D(a_{\text{occ}}),\, D(a_{\text{clean}} \odot (1-M))\big),$$
where $a_{\text{occ}}$ and $a_{\text{clean}}$ are the attention maps of the occluded and original image and $W$ is the Wasserstein distance between diagrams. The loss asks two things. Attention on the occluded image should keep the body's visible pieces at the right heights, and it should not create new foreground components on the occluder.
The height fusion is what makes this loss local. Without it, a spurious occluder blob can be matched to an unrelated true component. This is precisely the failure Oner et al. measured. The loss is added to the standard identity classification and triplet losses with weight $\beta$. Because $M$ is known, the target needs no human labels.
Visibility-gated matching. The distance between query $q$ and gallery $g$ combines the global feature distance with stripe distances weighted by joint visibility, $\sum_k v_k^q v_k^g\, d_k / \sum_k v_k^q v_k^g$. This needs no gallery reconstruction and no token pruning. It works in single-shot settings where FPC's consolidation has no unoccluded neighbour to draw on.
Analysis. We will measure how often the estimated visible extent is correct on synthetically occluded test images, where the ground truth is known. We will transplant Oner et al.'s monotonicity test by injecting occluders and checking whether the visibility estimate drops in the right stripes. We will also sweep $\lambda$ to locate where height fusion helps, with $\lambda=0$ as the standard-persistence control.
Baselines
Plain backbone. The naive reference point is the same ViT-B/16 trained with identity and triplet losses and matched on the global feature only, under identical augmentation including synthetic occluders. Synthetic occlusion augmentation alone is a known source of gains. Giving it to every baseline ensures the topology term is not credited for the augmentation. Beating it means a clear improvement in both Rank-1 and mean average precision (mAP) on Occluded-Duke, the benchmark with real occluders in a Duke-campus setting.
FPC. The strongest existing method with numbers in our file on that benchmark is FPC, which reports 76.7% Rank-1 and 72.8% mAP on Occluded-Duke. It shares our backbone and also derives occlusion reasoning from attention, so it is the most direct competitor. We will rerun it from released settings at the same resolution, epochs, and tuning budget, with matched hyperparameter search trials for both methods. We will compare both against the published number and against our rerun. Beating it means exceeding the rerun FPC on Occluded-Duke mAP without using any gallery reconstruction. Matching it while being gallery-independent would still be a meaningful result, which we would report as such.
Other occluded and holistic references. On Occluded-ReID, which uses mobile-camera captures of occluded pedestrians, the reference is HOReID at 80.3% Rank-1. ProFD reports gains on that benchmark but no absolute number in our file. On P-DukeMTMC, a partial-body benchmark derived from DukeMTMC, the reference is ProFD at 91.7% Rank-1. On holistic Market-1501, ProFD reports 95.1% Rank-1 and HOReID 94.2%. There we aim only to show no regression, since topology should matter little when nothing is occluded. DRFormer reports 85.5% mAP and 92.5% Rank-1 on DukeMTMC; it is evaluated on Occluded-Duke, but no reported result was found on Lacuna for that benchmark. Because it uses two foundation models, we treat it as a capacity-unmatched reference rather than a fair head-to-head.
Key ablations. Three ablations isolate the topological ingredient. The first sets $\lambda=0$, giving standard persistence without height. The second replaces the persistence-derived visibility with a row projection of thresholded attention. The third replaces it with single-threshold connected-component labelling. Each ablation keeps the stripe gating and the synthetic occluders, so only the visibility estimator changes.
Why not just…?
Why not just use a pose estimator to decide which parts are visible? This is HOReID's design, and that paper acknowledges that its accuracy is bounded by the HR-Net keypoints, which degrade under the heavy occlusion and crowding that define the task. ProFD likewise reports that off-the-shelf pose and parsing models produce noisy boundaries on re-identification data. We will run a keypoint-confidence visibility baseline through the same stripe gating. We expect it to fall behind specifically on images where another pedestrian is the occluder, since the detector then returns confident joints of the wrong person.
Why not just threshold attention and take a row-wise profile, or label connected components? This is the real threat, and it is why both appear as ablations. A row profile cannot tell whether foreground mass at a given height belongs to the target or to an adjacent distractor. Connected-component labelling can, but only at one threshold. That threshold is brittle across cameras, where attention contrast varies, and it is not differentiable, so it cannot drive the consistency loss. Persistence is the threshold-free, differentiable version of component labelling. If labelling at a tuned threshold matches our inference-time gains, we will say so. The remaining contribution would then have to come from the training loss, which the ablation of $\beta$ tests separately.
Why not just train a small head to predict stripe visibility from synthetic occlusion labels? The labels are free, so this is a strong and cheap alternative. Our hypothesis is that a supervised head learns the appearance of the pasted occluders and transfers poorly to real ones. The topological target constrains only the shape of attention, which should be less specific to occluder appearance. The decisive test trains both methods on Market-1501 with synthetic occluders and evaluates on Occluded-ReID, whose occluders are real and unseen. If the learned head transfers as well, the topological prior is unnecessary and we will report that.
Why not just fuse local and global foundation models, as DRFormer does? DRFormer shows that naive DINO-plus-CLIP concatenation is suboptimal and that balancing the two branches needs extra regularizers and roughly double the compute. It also gives no explicit handling of occluders beyond what attention learns. Our method is orthogonal and single-backbone. As a secondary experiment, we will add our gating on top of a DINO-initialized backbone to check whether the gains survive stronger features.
Proposed Experiments
The main study trains on Occluded-Duke and reports Rank-1 and mAP against the plain backbone, the rerun FPC, and the three ablations, over three seeds with confidence intervals. The transfer study trains on Market-1501 and tests on Occluded-ReID and P-DukeMTMC. This is where the synthetic-to-real argument against the learned visibility head is settled. Market-1501 itself checks for holistic regression, and MSMT17 serves as a larger-scale sanity check.
The diagnostic study uses held-out synthetic occlusions with known masks. It reports stripe-visibility accuracy and the monotonicity rate: the fraction of injected occluders that lower visibility in the correct stripes. This is reported for $\lambda=0$ and $\lambda>0$, a direct analogue of Oner et al.'s 40% failure measurement. We will report wall-clock overhead per batch to confirm the union-find cost is negligible on 128 tokens.
The proposal succeeds if three conditions hold. The height-fused model must beat the rerun FPC on Occluded-Duke mAP without gallery reconstruction. It must beat both the connected-component and learned-head visibility baselines on Occluded-ReID transfer. And $\lambda>0$ must clearly beat $\lambda=0$ in monotonicity. It fails, informatively, if the cheap visibility estimators match it on real occluders.
References
- Localized Persistent Homologies for more Effective Deep Learning
- Persistent Homology With Improved Locality Information for More Effective Delineation
- The fiber of the persistence map for functions on the interval
- TopoImages: Incorporating Local Topology Encoding into Deep Learning Models for Medical Image Classification
- High-Order Information Matters: Learning Relation and Topology for Occluded Person Re-Identification
- Dynamic Feature Pruning and Consolidation for Occluded Person Re-Identification
- ProFD: Prompt-Guided Feature Disentangling for Occluded Person Re-Identification
- DRFormer: A Dual-Regularized Bidirectional Transformer for Person Re-identification