Surveillance anomaly detectors usually judge motion through optical flow, the per-pixel displacement between consecutive frames. Perspective distorts that signal. A person sprinting fifty meters from the camera moves only a few pixels per frame, while a person strolling near the lens moves dozens. Detectors trained on raw flow therefore tend to miss unusual events in the background and raise false alarms in the foreground. The usual fix is to rescale the flow, using image height, mask area or a predicted depth map as a stand-in for distance. That fix leaves a second problem untouched. Real anomaly datasets contain very few far-away anomalies, so nobody can measure how blind a detector is at distance, and no detector is trained to see it. Filling that gap by hand means staging anomalies in a 3D tool, which is the asset-creation bottleneck that synthetic benchmarks such as UBnormal still rely on.
This note proposes a pipeline that builds both pieces from each camera's own normal footage, with no manual assets. It first estimates a dense per-camera scale map, giving pixels per meter at each image location, from the heights of people who walk through the scene. It then reuses those same people as assets. They are cut out of normal clips, re-timed in physical units, resized with the scale map, and pasted back at controlled distances as speed, direction and loitering anomalies. One geometric object, the scale map, is used three ways: to normalize motion, to place synthetic actors so their size and speed are consistent with the scene, and to sort every evaluation by distance.
Core Claim
The far-field blind spot of flow-based anomaly detection comes from two sources: how motion is represented, and where in the frame the training and evaluation anomalies fall. The second source can be fixed automatically, per camera, by recycling the camera's own actors under a scale map learned from those actors. The testable prediction is about real anomalies. On real held-out anomalies grouped by distance, combining perspective normalization with depth-balanced synthetic anomalies should shrink the near-versus-far detection gap more than either part alone. It should also shrink that gap more than the strongest existing depth-aware detector.
The closest prior work falls into three groups.
The first group motivates the problem. A Hybrid Video Anomaly Detection Framework via Memory-Augmented Flow Reconstruction and Flow-Guided Frame Prediction builds detectors on flow reconstruction and flow-guided frame prediction, which is exactly where the background-miss problem appears. Multi-Task Learning based Video Anomaly Detection with Attention fuses MiDaS monocular depth with flow magnitude and direction through an attention module. It reaches 89.1% frame-level area under the ROC curve (AUC) on ShanghaiTech, but it degrades on CUHK Avenue, where cameras are mounted low and occlusion is heavy. Both papers change the representation. Neither changes where anomalies appear, and neither measures accuracy as a function of distance.
The second group consists of synthetic anomaly data. UBnormal hand-stages 660 anomalies in 29 Cinema4D scenes. It does not control the depth distribution of those anomalies, and it needs CycleGAN to bridge the visual gap to real footage. Our pipeline differs in three ways: it uses no manual assets, it works in the target camera's own geometry and appearance, and it places anomalies on an explicit distance schedule.
The third group supplies methods we reuse. From the vertical-coordinate weighting in Fudan University at TRECVID 2008 and the mask-area normalization in AnimalFormer we take the idea that observed object size can serve as a free distance proxy. We generalize it from a single linear function of image height to a dense map that can follow stairs and changes in ground level. From the trajectory statistics in NHK STRL at TRECVID 2011 we take the idea of learning uncalibrated scene geometry from accumulated motion. From GeoSim we take real-to-sim composition, meaning assets mined from logs and placed with geometric occlusion reasoning. GeoSim works with 3D vehicles and LiDAR. We need neither, because a fixed camera can reuse 2D actor cutouts taken from its own viewpoint.
Research Program
Stage one: the scale map. We run a person detector and a segmentation model over each scene's normal training video. We record each person's pixel height together with the image location of their feet. We then fit a smooth field $s(x,y)$, the pixels per meter at foot location $(x,y)$, assuming an average adult height. The fit uses a spatially regularized regression so that regions with sparse observations borrow strength from their neighbors. Where people cross genuinely non-planar structure, the field is free to bend, which a single vertical-coordinate ramp cannot do. Normalized flow for a pixel belonging to an object whose ground contact is at $(x,y)$ is $\mathbf{v}_{\text{pixel}}/s(x,y)$, which is approximately meters per frame.
Stage two: anomaly synthesis. We segment and track walkers in normal clips and store each as a sprite sequence with its trajectory. To create an anomaly, we choose a target distance band and a target ground path inside it. We resize the sprite by the ratio of scale-map values at the source and target locations. We then change its physical speed in one of three ways: frame decimation produces speed anomalies, path reversal against the scene's dominant flow produces direction anomalies, and frame repetition with a stationary root produces loitering. The sprite is composited with depth-ordered occlusion, using foreground masks of real people already in the frame, and harmonized at the boundary.
This synthesis is deliberately limited to kinematic anomalies. That is the class where perspective matters and where pasted sprites are most believable. We do not attempt appearance anomalies such as vehicles on a walkway. Re-timed walking is not the same as running, so we treat synthetic speed anomalies as a proxy and validate everything on real anomalies.
Stage three: training and evaluation. We train flow-based detectors on normal data together with synthetic anomalies sampled uniformly across distance bands, in either a weakly supervised or an outlier-exposure setting. We evaluate on real test anomalies. Each real anomaly is assigned to a distance band using the scale map at its annotated location.
Baselines
The naive reference point is a flow-based detector trained on raw, unnormalized flow and normal data only. It shows the size of the far-field blind spot that every other method must close.
The strongest existing competitor is the depth-attention model of Multi-Task Learning based Video Anomaly Detection with Attention. It is the most direct published attack on perspective in this setting, and the paper reports 89.1% frame-level AUC on ShanghaiTech and 97.8% on UCSD-Ped2. We will reproduce it on our data splits and use the reproduced numbers rather than the reported ones. Our own variants will use the same backbone, the same number of training epochs and the same hyperparameter search budget. Beating it means two things together: matching its overall AUC within about one point, and clearly reducing its near-versus-far AUC gap on real anomalies.
The cheap geometric heuristics are vertical-coordinate scaling as in Fudan University at TRECVID 2008 and per-object mask-area scaling as in AnimalFormer. Each is applied to the same detector.
The synthetic-data reference is augmentation with UBnormal. The UBnormal paper reports 83.7% AUC on ShanghaiTech and 93.0% on Avenue for its augmented models. We will rerun this augmentation on our backbone so the comparison is with matched models.
Two ablations isolate the key ingredients. The first uses the scale map for normalization only, with no synthetic anomalies. The second trains with synthetic anomalies placed at the depths where real people naturally appear, which removes depth balancing. A third control pastes sprites without scale-map resizing or re-timing, which tests whether geometric consistency matters at all.
Why not just…?
Why not just normalize flow with the pedestrian-height scale map and skip synthesis? This is the real threat. Normalization alone might close most of the gap, because it is the representation fix and it is nearly free. However, normalization cannot create far-field training signal. It also cannot tell us whether a detector actually sees far-away anomalies, because real far-field anomalies are rare. The normalization-only ablation settles this directly. If it closes the real far-field gap as well as the full pipeline does, the synthesis contribution reduces to a benchmarking tool, and we will report it that way.
Why not just feed a monocular depth map as an extra input? The reported evidence cuts against relying on this alone. In Multi-Task Learning based Video Anomaly Detection with Attention, the motion branch rose from 69.7% to 79.19% AUC only with an explicit attention design. The same paper reports failure on Avenue's low, occluded views. Raw relative depth also lacks metric scale, so it cannot express "meters per frame." We include this model as the main baseline and add a variant that replaces MiDaS with our scale map, so that the two scale sources are compared within an identical architecture.
Why not just use domain randomization or an existing synthetic set like UBnormal? Randomized or hand-staged scenes do not share the target camera's geometry, and the parameter ranges cover only what an engineer thought to vary, as Pushkal Katara et al. argue for simulation generally. There is a suggestive gap in the reported numbers: UBnormal augmentation reaches 83.7% on ShanghaiTech against 89.1% for the in-domain depth model. Those results come from different backbones, so the gap is only suggestive, and our matched rerun is what will test it.
Why not just synthesize anomalies with a flow-conditioned video diffusion model? OnlyFlow conditions AnimateDiff on raw RAFT flow, and its authors acknowledge that it cannot separate camera motion from object motion or adapt to depth. A flow field moved to a new distance band would therefore carry the wrong pixel speeds. Its reported 54.5% to 71.1% preference wins measure visual appeal, not geometric correctness. Sprite reuse is cheaper, and its geometry is correct by construction.
Proposed Experiments
Data. The main experiments use ShanghaiTech, with its 13 fixed scenes and pixel-level anomaly masks, and CUHK Avenue. Avenue serves as the low-camera stress test where depth attention is reported to fail. UCSD-Ped2 serves as a near-saturated sanity check.
Metrics. We report frame-level AUC overall and within three distance bands defined by the scale map. We also report the far-minus-near AUC difference and the false-alarm rate on near-field normal motion.
Decisive check. The key result is the depth-stratified AUC on real test anomalies. A gain that shows up only on synthetic anomalies does not count. The idea succeeds if the full pipeline narrows the far-field gap beyond the normalization-only ablation, beyond the depth-attention model and beyond UBnormal augmentation, while keeping overall AUC competitive.
Supporting checks. We will estimate how much of the small real far-field anomaly set is needed to observe a reliable difference, and report confidence intervals with resampling across scenes. As a realism probe, a classifier will be trained to tell pasted actors from real ones. If it succeeds easily, detectors might be learning compositing artifacts rather than motion. We will test that possibility by checking whether gains survive when synthetic anomalies are replaced with normal-speed pasted sprites labeled as normal. We will also test whether the scale map beats the vertical-coordinate ramp in scenes with visible stairs or changes in ground level, which ShanghaiTech's scene variety should let us inspect case by case.
A negative outcome would also be informative. If depth balancing does not help on real anomalies, the blind spot is purely a representation problem, and the community's existing normalization tools are sufficient once they are evaluated by distance band.
References
- A Hybrid Video Anomaly Detection Framework via Memory-Augmented Flow Reconstruction and Flow-Guided Frame Prediction
- Multi-Task Learning based Video Anomaly Detection with Attention
- UBnormal: New Benchmark for Supervised Open-Set Video Anomaly Detection
- Fudan University at TRECVID 2008
- AnimalFormer: Multimodal Vision Framework for Behavior-based Precision Livestock Farming
- NHK STRL at TRECVID 2011: Surveillance Event Detection and Semantic Indexing
- GeoSim: Realistic Video Simulation via Geometry-Aware Composition for Self-Driving
- Gen2Sim: Scaling up Robot Learning in Simulation with Generative Models
- OnlyFlow: Optical Flow based Motion Conditioning for Video Diffusion Models