TL;DR
This paper introduces a novel co-training framework to accurately segment merged moving objects (like vehicles in a traffic scene) into individual instances. By leveraging two independent sets of features—scene context and appearance—the method effectively overcomes the limitations of traditional background subtraction methods that struggle with occlusions and close proximity of objects.
What Problem Does This Paper Address?
In video surveillance, a fundamental challenge is accurately detecting and tracking moving objects. A common issue arises when multiple objects, such as vehicles, are close together or partially occlude each other, causing them to be detected as a single, larger "merged" foreground object (a "multi-vehicle blob"). This merging makes it difficult to extract individual object features for subsequent tasks like classification and tracking. Existing methods often struggle to disentangle these merged blobs, especially when the individual objects share similar visual characteristics. This paper aims to solve this by first classifying blobs as either single- or multi-vehicle, and then segmenting the multi-vehicle blobs into their constituent single-vehicle components.
Methods
The proposed solution involves a two-step approach:
- Multi-vehicle Blob Classification: This step uses an unsupervised co-training framework to classify foreground blobs.
- Co-training: This machine learning paradigm trains two classifiers on different "views" (feature sets) of the data. Each classifier learns from the confident predictions of the other, allowing them to improve each other without requiring extensive manual labeling.
- Feature Sets: * Scene Context Features: These include the direction of motion and width distribution of vehicles within specific blocks of the scene. These are learned using Gaussian Mixture Models (GMMs) applied to vehicle trajectories and width measurements.
- Appearance Features: These are based on Multi-block Local Binary Patterns (MB-LBP), which capture texture and local image structures at multiple scales. An AdaBoost algorithm is used to select salient MB-LBP features and train a binary classifier.
- The outputs of these two classifiers are combined to make the final classification decision.
- Multi-vehicle Blob Segmentation: For blobs identified as multi-vehicle, a segmentation method is applied.
- Projection Histogram Analysis: This technique uses the vertical projection histogram of the blob, projected onto a direction perpendicular to the estimated motion direction. By analyzing the troughs in this histogram and considering the scene context features (like mean vehicle width), the algorithm segments the multi-vehicle blob into individual vehicle blobs.
Datasets
The paper does not explicitly mention specific named datasets for training or evaluation. Instead, it describes collecting "vehicle tracked sequences from 8 different scenes" and mentions that the training set consists of "79,175 single-vehicle blobs and 12,808 multi-vehicle blobs." The samples were collected over 10 frames to reduce correlation and represent diverse camera viewing angles.
Key Insights and Ideas
- Leveraging Independent Views: The core idea of co-training is powerful here. By using distinct feature sets (scene context vs. appearance), the method can overcome the limitations of each individual feature set. Scene context is robust to appearance variations but can be ambiguous on its own, while appearance features are sensitive to lighting and pose changes.
- Scene Context as a Prior: Using learned scene context (like motion direction) as a prior for segmentation is crucial. It helps disambiguate blobs that might look similar based on appearance alone.
- Unsupervised Learning for Efficiency: The unsupervised nature of the co-training approach significantly reduces the need for manual annotation, making the system more practical for real-world surveillance applications where large amounts of unlabeled data are readily available.
- Adaptive Learning: The use of online GMMs for learning scene context features allows the system to adapt to changes in the scene over time.
Key Results
- Classification Performance: The paper reports a classification accuracy of 91.59% for multi-vehicle blobs and 94.09% for single-vehicle blobs on a test set of 988 tracked sequences.
- Segmentation Performance: The segmentation module achieves a correct rate of approximately 95.0% for multi-vehicle blobs, with 10,797 out of 11,365 multi-vehicle blobs being segmented correctly.
- Real-time Performance: The segmentation framework is reported to be real-time, with processing times less than 0.1 seconds per frame on a 320x240 resolution image on a P4 3.2GHz PC.
Limitations and Future Work
- Shadows: The authors acknowledge that disturbances like shadows can affect the performance of the segmentation.
- Future Work: They suggest that improving object detection to better handle shadows would be a direction for future improvement.
Summary updated