View Independent Object Classification Based on Automated Ground Plane Rectification for Traffic Scene Surveillance

Zhaoxiang Zhang, Min Li, Kaiqi Huang, Tieniu Tan2008

TL;DR

This paper tackles the challenge of classifying objects (pedestrians, bicycles, vehicles) in traffic surveillance videos, which is notoriously difficult due to varying camera angles. They introduce a novel method for "ground plane rectification" that normalizes object features, making them independent of viewpoint. This allows for a robust online learning framework that achieves high classification accuracy without manual labeling.

What Problem Does This Paper Address?

Classifying objects in video surveillance, especially in traffic scenes, is crucial for intelligent monitoring (e.g., detecting forbidden area intrusions). However, a major hurdle is "perspective distortion." Cameras capture objects at different angles, causing their apparent size, speed, and shape to vary significantly. This makes traditional 2D features unreliable for accurate classification. Existing methods often rely on manual camera calibration, which is impractical for diverse scenes, or struggle with low-resolution videos and the sheer variety of object appearances. The authors aim to overcome this by making object features viewpoint-invariant.

Methods

The core of their approach lies in automated ground plane rectification. They leverage the common observation that most traffic objects move on the ground plane.

  1. Ground Plane Rectification:
    • Appearance and Motion Information: They use both visual cues (like object silhouettes) and motion data from detected vehicles to estimate the ground plane.
  2. Homography Decomposition: The relationship between the 3D ground plane and the 2D image plane is modeled by a homography matrix. They decompose this matrix into components that allow for both affine (preserving parallelism) and metric (preserving distances and angles) rectification.
  3. Vanishing Points: A key step in affine rectification is identifying "vanishing points" – points where parallel lines in 3D appear to converge in the 2D image. They propose methods to robustly estimate these points from vehicle motion and orientation, even in complex roadway layouts (straight segments, crossroads, bends).
  4. Metric Rectification: This step further refines the rectification by using constraints from known angles on the ground plane, effectively recovering the true geometric relationships.
  5. Online Object Classification:
    • Feature Normalization: Once rectified, features like object size, speed, and compactness become independent of the camera's viewpoint.
  6. Unsupervised K-Means Clustering: Initially, they use K-Means clustering on the rectified features (compactness, size, angle) to automatically group objects into three categories (pedestrian, bicycle, vehicle) without manual labels.
  7. Bayesian Classification: A Bayesian classifier is then trained using these clusters. The classifier's parameters (mean and covariance of feature distributions) are continuously updated online as new data arrives, making it robust to changing conditions.
  8. Refinement: The classification results can also feedback to improve the ground plane rectification, creating a synergistic loop.

Datasets

Not explicitly detailed in the paper. The experiments appear to be conducted on traffic surveillance videos, with the authors mentioning "four traffic scenes of different view angles" for testing. No specific public benchmark datasets are listed.

Key Insights and Ideas

  • Ground Plane as a Unifying Constraint: The paper highlights that the ground plane is a powerful, often overlooked, constraint that can be exploited to overcome perspective distortion in traffic scenes.
  • Automated Rectification is Key: Moving away from manual camera calibration towards automated methods using object motion and appearance is crucial for practical, scene-agnostic surveillance systems.
  • Viewpoint-Invariant Features: The core idea is that by rectifying the ground plane, features derived from objects become invariant to camera viewpoint, significantly simplifying the classification task.
  • Online Learning for Adaptability: The use of an online learning framework allows the system to adapt to new environments and changing conditions without requiring retraining on large, pre-labeled datasets.
  • Synergy between Rectification and Classification: The feedback loop where classification results improve rectification, and vice-versa, demonstrates a robust system design.

Key Results

  • Improved Feature Separation: Figure 10 clearly shows that rectified features (size, speed, compactness) exhibit much better separation between classes (pedestrian, bicycle, vehicle) compared to original, unrectified features.
  • High Classification Accuracy:
    • With rectified features, they achieve an average classification accuracy of 99.1% for Pedestrians, 94.7% for Bicycles, and 98.2% for Vehicles (Table 1).
  • This is significantly better than classification using original features without rectification, which yields lower accuracies (Table 2).
  • Their method also outperforms a previous approach ([15]) that uses scene division for perspective handling, achieving better results by directly addressing perspective distortion through rectification (Table 3).
  • Real-time Performance: The algorithm can process videos at 15 frames per second, which is considered real-time performance.

Limitations and Future Work

  • Degenerate Case (Top-Down View): The authors acknowledge that their approach is less effective in a pure top-down camera view, as perspective distortion is minimal in such cases. However, they argue that traffic surveillance typically uses oblique views, making this less of a practical issue.
  • Roadway Layout Assumptions: While they extend their method to handle various roadway layouts (straight, bends, crossroads), complex, highly dynamic intersections with many overlapping paths might still pose challenges.
  • Future Work: The paper suggests that the ground plane rectification technique can be extended to other applications like camera calibration, simplifying the process significantly. They also mention that temporal information from tracking can further improve classification robustness.

Summary updated

Abstract

We address the problem of view independent object clas-sification. Our aim is to classify moving objects of traf-fic scene surveillance videos into pedestrians, bicycles and vehicles. However, this problem is very challenging due to large object appearance variance, low resolution videos and limited object size. Especially, perspective distortion of surveillance cameras makes most 2D object features like size and speed related to view angles and not suitable for object classification. In this paper, we adopt the common constraint that most objects of interest in traffic scenes are moving on the ground plane. Firstly, we realize the ground plane rectification based on appearance and motion infor-mation of moving objects, which can be applied for normal-ization of 2D object features. An online learning framework is then described to achieve automatic object classification based on rectified 2D object features. Experimental results demonstrate the effectiveness, efficiency and robustness of the proposed method. 1.
BibTeX
@misc{zhang2008view,
  author = {Zhaoxiang Zhang and Min Li and Kaiqi Huang and Tieniu Tan},
  title = {{View Independent Object Classification Based on Automated Ground Plane Rectification for Traffic Scene Surveillance}},
  year = {2008},
}
Figure