Imagine a camera overlooking a long street. A person running near the camera crosses 100 pixels in a single second. Far down the street, another person running at the exact same physical speed crosses only 5 pixels. To a naive computer vision model operating purely in pixel space, the first person appears to be moving twenty times faster than the second.
This discrepancy is the direct result of perspective projection—the mathematical mapping of a 3D world onto a 2D image plane. As noted by Hajar Sadeghi Sokeh et al. in Towards unsupervised semantic segmentation of street scenes from motion cues, perspective distortion causes an object's perceived size and speed to vary continuously with its distance from the lens. Without accounting for this distortion, uniform motion analysis across a scene becomes impossible.
To build models that truly understand physical movement, researchers use perspective-invariant motion normalization. This technique scales raw pixel displacements based on depth cues, transforming viewpoint-dependent optical flow into a consistent representation of physical velocity.
Why Raw Motion Fails Downstream Tasks
When computer vision models rely on unnormalized pixel displacement or raw optical flow, perspective distortion introduces severe spatial biases. This failure manifests across several critical domains:
- Video Anomaly Detection: Anomaly detection models flag unusual or high-speed events. If a model does not account for depth, a normal, fast-moving object in the foreground might be falsely flagged as an anomaly. Conversely, a genuinely dangerous, high-speed event in the background will produce a tiny optical flow magnitude and go completely unnoticed. Yongwei Nie et al. highlight this vulnerability in A Hybrid Video Anomaly Detection Framework via Memory-Augmented Flow Reconstruction and Flow-Guided Frame Prediction, demonstrating that a lack of depth normalization causes background anomalies to be systematically overlooked.
- Action and Behavior Recognition: For surveillance and behavioral analysis, absolute pixel thresholds fail. Masaki Takahashi et al. point out in NHK STRL at TRECVID 2009: Surveillance Event Detection and High-Level Feature Extraction that distance-based bias leads to highly inconsistent classification of human actions across a camera's field of view. Similarly, in precision livestock farming, Ahmed Rashid Qazi et al. observe in AnimalFormer: Multimodal Vision Framework for Behavior-based Precision Livestock Farming that raw velocity measurements are useless for comparative behavioral analysis unless scaled by the animal's relative distance.
- Compressed Domain Processing: To save compute, many real-time systems extract motion vectors directly from compressed video streams (such as MPEG or H.264 bitstreams) without fully decoding the frames. As explored by Costantino Grana et al. in Compressed Domain Features Extraction for Shot Characterization, modern video codecs utilize variable block sizes and multi-frame references, making the underlying motion metadata highly inconsistent. When perspective distortion is added to this mix, the raw compressed features become too distorted for reliable analysis.
Mechanisms of Motion Normalization
To resolve these issues, researchers have developed several clever mechanisms to normalize motion vectors, depending on the available data and the scene context.
[Raw Pixel Motion] ───► [Apply Depth/Scale Proxy] ───► [Perspective-Normalized Motion]
│
├──► Vertical Coordinate (y-axis)
├──► Object Height / Mask Area
└──► Explicit Depth Map
Geometric and Vertical Coordinate Scaling
In many fixed surveillance setups, the ground plane recedes into the distance along the vertical axis of the frame. This means the vertical ($y$) coordinate of an object can serve as a direct proxy for its depth.
Xiangyang Xue et al. leverage this geometric property in Fudan University at TRECVID 2008 (TRECVID 2008) by weighting motion vectors using a vertical coordinate coefficient. This simple linear or polynomial adjustment automatically upscales distant movements, ensuring that background actions are weighted equally with foreground actions.
Statistical Trajectory Clustering
When explicit camera calibration is unavailable, scene geometry can be inferred statistically from the video itself. In NHK STRL at TRECVID 2011: Surveillance Event Detection and Semantic Indexing, Masaki Takahashi et al. normalize motion descriptors using relative thresholds derived from the mean and standard deviation of movement within localized trajectory clusters. By evaluating motion relative to historical patterns in that specific region of the frame, the system naturally compensates for perspective speed variations.
Object Scale and Height Estimation
Another powerful approach infers depth by tracking how an object's apparent size changes as it moves through the scene.
In Group activity recognition on outdoor scenes and Human group activity recognition based on modelling moving regions interdependencies, Kyle Stephens et al. utilize a two-step segmentation process. The first pass uses an Expectation-Maximization algorithm on a Gaussian Mixture Model to estimate the relative heights of moving objects across the scene. The second pass applies this height-based scaling factor to normalize perceived velocities, enabling robust group interaction analysis regardless of the camera viewpoint (Kyle Stephens et al., Recognizing Interactions Between People from Video Sequences).
A similar intuition applies to instance segmentation masks. Ahmed Rashid Qazi et al. normalize animal movement speed by dividing raw pixel displacement by the pixel area of the subject's segmentation mask. As an animal moves away and shrinks in pixel area, the denominator decreases, mathematically boosting the calculated velocity to match its true physical speed.
Explicit Depth Modeling and Contextual Anchors
With the advent of deep multi-task learning, models can now explicitly predict a scene's depth map to guide motion scaling. Mohammad Baradaran et al. demonstrate this in Multi-Task Learning based Video Anomaly Detection with Attention, where a context-aware motion branch integrates explicit depth estimation. This allows the network to dynamically adjust expected movement speeds based on distance, preventing normal foreground movement from being flagged as anomalous.
Alternatively, known physical structures in the environment can act as normalization anchors. In Phase Space Reconstruction Network for Lane Intrusion Action Recognition, Ruiwen Zhang et al. normalize object motion by the pixel width of the traffic lane at the object's current position. This transforms raw 2D video coordinates into a 1D relative motion time series, making the system completely invariant to both perspective distortion and the host vehicle's camera ego-motion.
Unlocking High-Fidelity Representations
By neutralizing perspective distortion, these normalization techniques turn noisy, viewpoint-dependent pixel data into high-fidelity representations of physical reality.
For instance, Xu Liu et al. demonstrate in Highway Vehicle Counting in Compressed Domain that when noisy compressed-domain metadata is normalized for temporal scales and perspective distortions, it can function as a high-fidelity representation of physical movement. This allows their system to achieve the accuracy of fully decoded pixel-domain models at a fraction of the computational cost.
The same underlying principle applies to advanced 3D reconstruction. In Robust e-NeRF: NeRF from Sparse & Noisy Events under Non-Uniform Motion, Gim Hee Lee et al. extend the concept of motion normalization to the temporal domain. By normalizing reconstruction losses by the target gradient, they make optimization invariant to non-uniform camera velocities.
Whether dealing with surveillance streams, animal tracking, or neural radiance fields, perspective-aware normalization bridges the fundamental gap between what a camera sensor records and how objects actually move through physical space.
Key Foundational Work
Fudan University at TRECVID 2008 (2008) — contributed 3 concept elements to this direction.
Compressed Domain Features Extraction for Shot Characterization (2007) — contributed 1 concept elements to this direction.