Map-based Active Leader-Follower Surveillance System

Himaanshu Gupta, Xiaochun Cao, Niels HaeringINRIA 2008

TL;DR

This paper introduces a novel framework for active leader-follower surveillance systems that can ingest data from various leader sensors (radar, AIS, cameras) to detect unusual events. The system then calibrates follower PTZ cameras to precisely track these events by decoupling the calibration into map-based and PTZ-specific steps, enabling scalable and robust surveillance.

What Problem Does This Paper Address?

Traditional leader-follower surveillance systems, where a wide-field-of-view leader camera directs PTZ (pan-tilt-zoom) follower cameras to track targets, suffer from limitations. A key issue is that follower cameras can only track targets as long as they remain within the leader's field of view. Furthermore, inaccuracies in leader-follower calibration can prevent precise zooming onto targets. Automating the follower camera's ability to actively track targets, even when they move out of the leader's view, and refining zoom for optimal visualization are critical challenges. This paper aims to address these by proposing a robust and scalable system that can handle diverse sensor inputs and perform accurate, automated PTZ control for continuous target tracking.

Methods

The proposed system breaks down the leader-follower calibration into two independent, crucial steps:

  1. Map-based calibration of leader cameras: This involves geo-registering static leader cameras to a world coordinate system. The paper utilizes a planar homography to map camera image coordinates to latitude/longitude values, leveraging existing geo-positioning information from mobile transponders. A robust data filtering mechanism is employed to handle noisy correspondences in uncontrolled environments.
  2. PTZ camera calibration to geo-locations: Once the leader camera is geo-registered, this step focuses on mapping latitude/longitude coordinates to the follower PTZ camera's pan-tilt-zoom settings. This is achieved by manually slewing the follower camera to known landmarks and recording the corresponding PTZ angles. The system then learns a planar homography to project ground plane points to a virtual plane, enabling accurate pan and tilt calibration. Zoom calibration is also addressed, though acknowledged as subjective.

Beyond calibration, the system incorporates:

  • Leader Sensor Processing: Rule and learning-based pattern recognition techniques are applied to leader sensor data (e.g., radar, AIS, cameras) to infer unusual behaviors and events. This includes improving background subtraction for dynamic scenes and using Gentle AdaBoost for target classification.
  • PTZ Camera Autocalibration: To handle the variability of commercial PTZ cameras, the paper proposes methods to automatically acquire PTZ parameters like focal length, zoom profile, and pan/tilt velocity profiles. This is done by sending sequences of PTZ commands and analyzing the resulting video.
  • Active Tracking: Once the PTZ camera is directed to a target, an active tracking algorithm takes over. This algorithm uses adaptively selected discriminative color subspace features and a mean-shift procedure to segment and track targets in subsequent frames, even with significant scale changes. A Kalman filter is used to estimate target location and velocity.
  • Active PTZ Controller: Instead of issuing absolute PTZ commands, the system employs a velocity-based controller for smoother and more continuous camera motion. This controller predicts camera position changes, computes desired PTZ corrections, and issues velocity commands based on learned PTZ profiles.

Datasets

Not explicitly discussed in the paper. The authors mention using a COTS Pelco Spectra III camera for experiments and employing synthetic data for quantitative analysis of calibration accuracy.

Key Insights and Ideas

  • Decoupling Calibration: The most significant insight is the separation of leader camera geo-registration from PTZ camera calibration. This modular approach allows for easier integration of diverse leader sensors (like Radar and AIS, which provide direct geo-location data) and simplifies the addition of new follower cameras, leading to improved scalability.
  • Automated PTZ Parameter Acquisition: The realization that manual calibration of commercial PTZ cameras is impractical drives the development of automatic methods for acquiring crucial parameters like zoom profiles and velocity characteristics. This is essential for robust and adaptive control.
  • Velocity-Based PTZ Control: Moving from absolute PTZ commands to velocity-based commands significantly enhances the viewing experience by providing smoother, more continuous camera motion, which is crucial for effective surveillance.
  • Adaptive Feature Selection for Tracking: The use of adaptively selected discriminative color subspace features allows the tracking algorithm to focus on the most informative visual cues, improving robustness in complex scenes and handling target appearance changes.

Key Results

  • Calibration Accuracy: The proposed leader-follower calibration method demonstrates accurate camera positioning, achieving an average pan/tilt error of less than one degree for almost all viewing angles in synthetic data experiments.
  • Scalability: The two-step calibration process allows for an expansion to 'n' leader and follower cameras with an O(n) calibration time, a significant improvement over O(n²) methods.
  • Event Detection: The system achieved high performance in event detection, with a precision of 88.71% and a recall of 98.21% for detecting tripwire events, with only 7 false alarms across 56 events.
  • PTZ Autocalibration: The autocalibration process successfully characterized PTZ camera parameters, including pan/tilt and zoom profiles, and estimated actuation delays (70ms for pan/tilt, 130ms for zoom).

Limitations and Future Work

  • Zoom Optimality: The paper acknowledges that the optimality of zoom is subjective and varies by application. While they provide a method for zoom calibration, further work could explore more sophisticated approaches to automatically determine optimal zoom levels for different scenarios.
  • Real-world Data: While synthetic data was used for calibration accuracy evaluation, the paper could benefit from more extensive quantitative results on real-world datasets for all components.
  • Target Complexity: The active tracking algorithm is noted to be effective for sparsely distributed targets like watercraft. Extending its robustness to scenarios with dense crowds or highly occluded targets would be a valuable future direction.
  • Sensor Fusion: While the framework supports multi-modal leader sensors, the paper focuses primarily on camera-based leader processing. Further research could explore deeper fusion strategies for enhanced event detection and target characterization.

Summary updated

Abstract

International audience
BibTeX
@inproceedings{gupta2008map,
  author = {Himaanshu Gupta and Xiaochun Cao and Niels Haering},
  title = {{Map-based Active Leader-Follower Surveillance System}},
  booktitle = {INRIA},
  year = {2008},
}
Figure