Abstract
This paper presents a robust multi-camera person detection method designed to handle complex occlusions by processing each video frame independently. Unlike traditional tracking systems that rely on temporal continuity, our approach utilizes a generative model to iteratively refine an occupancy "probability field" on the ground plane. By comparing synthetic expectations of binary silhouettes with actual observed motion blobs, the system effectively resolves occlusions without the risk of error propagation common in temporal trackers. We demonstrate the efficacy of this method on sequences involving multiple interacting individuals.
Introduction
Detecting and tracking people in crowded environments remains a fundamental challenge in computer vision, primarily due to the frequent occurrence of occlusions. In 2004, most state-of-the-art systems addressed this by leveraging temporal continuity through frameworks like Hidden Markov Models (HMMs) or Kalman filters. While these models help predict positions, they are prone to divergence if a tracking error occurs. Furthermore, they are often unsuitable for low-frame-rate surveillance systems where temporal coherence is minimal.
This research explores whether the occlusion problem can be solved using only the instantaneous information available across multiple camera views. We propose a "top-down" generative approach that treats detection as a global optimization problem within a single time frame.
Methodology
The core of our approach is the Fixed Point Probability Field. The process begins by partitioning the ground plane into a regular grid, where each cell $i$ is assigned a probability $\rho_i$ of being occupied by a person.
Generative Model and Synthetic Images
Instead of a direct "feed-forward" mapping from 2D image features to 3D space, we employ a generative model. Given a set of occupancy probabilities on the grid, we calculate the expected visual output for each camera. People are modeled as simple human-sized rectangles, which are projected into each camera view using homography. This allows us to compute an "average synthetic image"—an analytical expectation of what the binary silhouettes should look like based on the current probability field.
Fixed Point Iteration
The detection is framed as an iterative update process. The system compares the synthetic images to the actual binary motion blobs extracted from the video feed. If the presence of a person at a specific grid location improves the match between the synthetic and real images, the probability $\rho_i$ for that cell is increased.
This process typically stabilizes within approximately 100 iterations, reaching a "fixed point" that represents the most likely distribution of people on the ground plane. To maintain computational efficiency, we utilize integral images, allowing the distance between synthetic and real images to be calculated in constant time.
Experimental Evaluation
The proposed method was evaluated on two internal video sequences captured by synchronized cameras ($320 \times 240$ resolution) in a room mapped to 1,332 ground-plane locations.
- 2-People Sequence: This sequence consists of 280 frames featuring two individuals walking and occluding each other. The system achieved a false positive rate of 0.89% and a misdetection rate of 6.56%.
- 3-People Sequence: This more challenging 280-frame sequence involves three people with complex interactions. While the false positive rate remained low at 1.66%, the misdetection rate rose to 14.38% due to the increased complexity of the occlusions.
Qualitative results demonstrate that the system successfully locates individuals even when they almost entirely overlap in both camera views. The algorithm's processing speed was approximately 3.4 seconds per frame on a 1.4GHz Pentium-M processor.
Discussion and Key Insights
The "analysis by synthesis" approach provides inherent occlusion handling. Because the generative model accounts for the fact that a person in the foreground blocks the view of someone behind them, the algorithm naturally discounts information from occluded regions. If a location is hidden in one camera view, the system automatically relies more heavily on other views where the location remains visible.
Furthermore, frame independence makes the system immune to the "drift" typical of traditional trackers. If a person is lost for several frames, the system can recover instantly as soon as they become visible again. The global optimization over the probability field also provides robustness against noisy or broken binary blobs produced by primitive motion detectors.
Conclusion and Future Work
While effective, the system has limitations. It is heavily dependent on the quality of the underlying motion detection; if a person stops moving and is absorbed into the background model, the algorithm will fail to detect them. Additionally, the use of simple rectangular shapes is a coarse approximation that limits localization precision.
Future work involves integrating this robust frame-independent detection as an observation model within a larger temporal tracking framework, such as an HMM. This would combine the instantaneous robustness of the probability field with the smoothness and predictive power of temporal models.
Summary updated