Abstract
This paper presents a novel framework for learning a contextual motion model involving multiple objects in far-field surveillance video. We represent the trajectories of multiple objects using a 3D graph $\mathcal{G}$ in $(x, y, t)$ space, augmented by spatio-temporal relations between moving and static objects. An inhomogeneous Markov model is defined over $\mathcal{G}$, with parameters estimated via Maximum Likelihood Estimation (MLE) and relations pursued through a minimax entropy principle. This approach allows for the synthesis of new video sequences that reproduce observed statistics. The learned model is applied to improve object tracking and detect sophisticated abnormal events, such as traffic rule violations, through a log-likelihood ratio test estimated by importance sampling.
Introduction
Automated surveillance systems often struggle to track multiple similar-looking objects and identify complex abnormal behaviors. Traditional methods frequently treat objects in isolation or rely on simplistic motion models like constant velocity. However, real-world motion is deeply contextual; a vehicle's path is constrained by road geometry (static context), and its dynamics are influenced by other agents (dynamic context). Without accounting for these interactions, trackers often fail during occlusions, and abnormality detection remains limited to simple outliers like speeding.
This framework addresses these challenges by learning a "Trajectory Graph" that captures the complex spatio-temporal relationships between multiple moving objects and their environment.
Modeling Contextual Motion
The core of the proposed system is the Trajectory Graph ($\mathcal{G}$). In this representation, nodes represent object states, and edges represent four distinct types of relations:
- Object-to-Scene: Connections to semantic regions such as sources (entries), sinks (exits), and paths.
- Single-Object Temporal: Smoothness, speed, and acceleration along a single track.
- Multi-Object Spatial: Relative distance and direction between different objects within the same frame.
- Multi-Object Interactive: Complex interactions over time, such as vehicles yielding to pedestrians at crosswalks.
Learning through Relation Pursuit
The model is learned using a minimax entropy principle. Starting with a simple base model, the algorithm iteratively selects relations from a "Relation Library" that provide the highest information gain. This process minimizes the KL divergence between the model and the observed training data.
To validate the model, the authors employ an "analysis-by-synthesis" approach. A successful model should be capable of synthesizing entirely new, realistic traffic trajectories that match the statistical properties of the training data. As shown in Figure 1, as the number of relations ($n$) increases, the KL-divergence decreases, and the system's ability to detect sophisticated abnormal events improves.
Abnormality Detection Framework
Abnormality is defined using a log-likelihood ratio test. The system compares the probability of a specific trajectory subgraph under the learned contextual model versus a uniform "null" model. If a behavior is highly improbable given the learned "rules" of the scene, it is flagged.
Because detecting abnormalities in high-dimensional trajectory space is computationally intensive, the authors utilize importance sampling. As illustrated in Figure 6, the system samples "particles" from the learned model to estimate the expected behavior at any given time, allowing for efficient real-time abnormality measures and the generation of spatial-temporal heat maps.
Experimental Results
The model was evaluated using traffic surveillance sequences and a dataset of 5,000 trajectories containing synthetic abnormalities.
- Improved Tracking: By using the scene model as a prior for a Particle Filter, the tracker leverages social and spatial constraints. This prevents the "mode collapse" common in standard filters during occlusions, where multiple white vehicles might otherwise be confused.
- Sophisticated Detection: The system successfully identified complex events that simpler models miss, including "car moving off road," "collisions," and "traffic rule violations" (e.g., driving through a pedestrian-heavy crosswalk).
- Quantitative Performance: Precision-Recall and ROC curves demonstrated significant area-under-the-curve (AUC) improvements as contextual relations were added to the model.
Conclusion
By integrating environmental and interactive context into a unified trajectory graph, this framework significantly enhances the robustness of surveillance systems. While current limitations include the need for manual semantic annotation and high computational demands for massive object counts, the "relation pursuit" methodology provides a scalable path toward automated, context-aware scene understanding.
Summary updated