Abstract
Traffic Surveillance Systems (TSS) have become increasingly crucial in modern intelligent transportation systems, with vision technologies playing a central role for scene perception and understanding. While existing surveys typically focus on isolated aspects of TSS, a comprehensive analytical framework bridging low-level and high-level perception tasks, particularly considering emerging technologies, remains lacking. This paper presents a systematic review of vision technologies in TSS, examining both low-level perception tasks (object detection, classification, and tracking) and high-level perception tasks (parameter estimation, anomaly detection, and behavior understanding). Specifically, we first provide a detailed methodological categorization and comprehensive performance evaluation for each task. Our investigation reveals five fundamental limitations in current TSS: perceptual data degradation in complex scenarios, data-driven learning constraints, semantic understanding gaps, sensing coverage limitations and computational resource demands. To address these challenges, we systematically analyze five categories of current approaches and potential trends: advanced perception enhancement, efficient learning paradigms, knowledge-enhanced understanding, cooperative sensing frameworks and efficient computing frameworks, critically assessing their real-world applicability. Furthermore, we evaluate the transformative potential of foundation models in TSS, which exhibit remarkable zero-shot learning abilities, strong generalization, and sophisticated reasoning capabilities across diverse tasks. This review provides a unified analytical framework bridging low-level and high-level perception tasks, systematically analyzes current limitations and solutions, and presents a structured roadmap for integrating emerging technologies, particularly foundation models, to enhance TSS capabilities.
Introduction: The Need for a Holistic Framework
While Intelligent Transportation Systems (ITS) have advanced rapidly, Traffic Surveillance Systems (TSS)—the "eyes" of these systems—often suffer from fragmented development. Most research focuses on isolated tasks: one paper might solve vehicle counting, while another tackles accident detection, but they rarely integrate into a cohesive system.
This survey addresses the lack of a holistic framework that connects low-level perception (identifying objects) with high-level understanding (interpreting intent). It identifies five fundamental "pain points" in the industry:
- Perceptual Degradation: Performance drops in adverse weather (rain, fog) or low light.
- Data Constraints: The massive hunger for expensive, manually labeled data.
- Semantic Gaps: AI can detect a car but lacks the "common sense" to understand complex traffic interactions.
- Coverage Limits: Blind spots and the difficulty of tracking objects across multiple cameras.
- Compute Demands: The challenge of running heavy AI models on roadside "edge" hardware.
Methodological Categorization
Vision technologies in TSS are categorized into a two-tier hierarchy:
- Low-Level Tasks: 2D/3D object detection, fine-grained classification (identifying make/model), and multi-object tracking (MOT).
- High-Level Tasks: Traffic parameter estimation (speed/density), anomaly detection (accidents/violations), and behavior understanding (predicting pedestrian crossing).
The field has shifted from traditional "hand-crafted" algorithms to modern Deep Learning (CNNs and Vision Transformers). A major emerging focus is Foundation Models (FMs). Unlike task-specific models, FMs are pre-trained on massive datasets, allowing for "zero-shot" tasks—identifying rare vehicle types never explicitly seen during training—and providing reasoning capabilities that help bridge the semantic gap.
Benchmarking and Datasets
The survey provides a directory of benchmarks across various tasks:
- Detection/Tracking: UA-DETRAC, SEU_PML, and the 3D-focused Rope3D and DAIR-V2X.
- Re-Identification: VeRI-776 and CityFlow (essential for city-wide tracking).
- Anomaly Detection: UCF-Crime and traffic-specific CADP and CDD.
- Behavior Prediction: ETH/UCY for pedestrians and JAAD/PIE for crossing intentions.
While datasets for autonomous driving are plentiful, there remains a scarcity of high-quality, long-range 3D surveillance datasets specifically tailored for TSS.
Key Insights and Results
Synthesis of performance across hundreds of methods reveals several trends:
- 2D Detection: One-stage detectors (like the YOLO series) have become the standard for real-time TSS due to their balance of speed and accuracy.
- 3D Detection: "Direct estimation" methods (learning 3D boxes directly from 2D images) are now outperforming traditional geometric methods that rely on vanishing points.
- Anomaly Detection: Unsupervised methods, which learn "normal" patterns and flag deviations, are becoming more practical than supervised methods as they don't require labels for every possible accident type.
A notable insight is the "Perception-Performance Gap": cleaning up an image (e.g., de-raining) doesn't always improve detection. Sometimes, artifacts from enhancement models confuse the detector. Furthermore, behavior prediction is moving from rigid physics-based models toward social intelligence models (GNNs and Transformers) that account for interactions between road users.
Future Outlook: Foundation Models and Cooperative Sensing
The future of TSS lies in addressing current limitations through emerging technologies:
- Foundation Models as "Common Sense": Large Vision Models can provide context, explaining why an anomaly occurred (e.g., mechanical failure vs. traffic jam).
- World Models for Data Scarcity: Generative models (like Sora) can create realistic synthetic videos of rare accidents to solve data imbalance.
- Edge Deployment: Future work must focus on model distillation and lightweight architectures to run Foundation Models on roadside hardware.
- Cooperative Sensing (V2X): Integrating roadside cameras, vehicle sensors, and cloud computing to form a unified "digital twin" of the urban environment.
Summary updated