Abstract
Catadioptric sensors, which combine cameras and mirrors to provide a 360-degree field of view, are invaluable for robotics and surveillance. However, they introduce significant geometric distortions and are highly sensitive to lighting changes. This paper proposes a robust tracking method that adapts a conventional color particle filter to operate on a unit sphere. By redefining tracking windows in spherical coordinates and employing geodesic distances, the method effectively mitigates the non-linear warping of objects. Experimental evaluations across diverse indoor and outdoor sequences show that the spherical adaptation significantly improves tracking persistence and spatial accuracy over standard image-plane trackers.
Introduction
Catadioptric vision systems offer a massive field of view with a single sensor, making them ideal for wide-area surveillance and autonomous navigation. Despite these advantages, the images produced are subject to a "distortion tax." Because light reflects off a curved mirror before reaching the camera sensor, straight lines appear curved, and the resolution of objects varies non-linearly depending on their position in the panoramic view.
Standard visual tracking algorithms, typically designed for perspective cameras, rely on rectangular search windows and Euclidean distance metrics. These assumptions fail in catadioptric imagery, where an object's shape and size change drastically as it moves across the field of view. Furthermore, the mirrors used in these systems are prone to reflections and intense glare, which can easily confuse simple color-based trackers.
Methodology
To address these challenges, we propose a "spherical" adaptation of the conventional color particle filter. The core innovation lies in shifting the mathematical domain from the distorted 2D image plane to a 3D unit sphere, which represents the natural geometry of the catadioptric system.
Spherical Neighborhood and Geodesic Distance
Instead of defining a tracking window as a fixed rectangle of pixels, we project the image onto a unit sphere. The tracking "neighborhood" is defined by angular ranges—latitude ($\phi$) and longitude ($\theta$). When this spherical window is back-projected onto the 2D image, it naturally warps to match the sensor's specific distortion. To account for the non-linear resolution across the mirror, we utilize geodesic distance (the shortest path along the surface of the sphere) for kernel density estimation rather than standard pixel-based Euclidean distance.
Robust Target Representation
To improve robustness against background clutter and scale changes, the target is represented by a multi-part color histogram. We utilize seven distinct histograms: one for the entire patch, four for quadrants (to capture rotation), and two for the inner and outer areas (to handle scale variations). This "spatial fingerprint" ensures that the tracker can distinguish the target even when the background contains similar colors.
Illumination Invariance
To combat the mirror's sensitivity to light shifts and reflections, we employ a transformed RGB color space. Each channel is adjusted by its mean and standard deviation, providing a degree of invariance to illumination changes. The particle filter's state vector is defined by spherical coordinates $(\theta, \phi)$, their respective velocities, and the angular window size $(\delta\theta, \delta\phi)$, allowing the tracker to maintain a consistent understanding of movement in a 360-degree context.
Datasets
The proposed approach was evaluated using four custom catadioptric video sequences (640x480 resolution), each designed to test specific environmental challenges:
- Outdoor: Tracking a person using a non-central camera configuration.
- Indoor (Reflective): Tracking an object near highly reflective surfaces with significant scale variation.
- Outdoor (Dazzlement): Tracking a person through intense glare and direct sunlight.
- Indoor (Occlusion): Tracking an object through periods of total occlusion.
Sensor calibration was performed using the 'OCamCalib' toolbox, and manual ground-truth data was generated for quantitative comparison.
Results and Discussion
The adapted spherical particle filter significantly outperformed conventional image-plane methods across all test scenarios.
- Tracking Persistence: In the reflective indoor sequence (Sequence 2), the conventional tracker lost the target after only 44% of the video. In contrast, the adapted method maintained a successful track for 99.7% of the 649 frames.
- Spatial Accuracy: The "center distance" error—the gap between the estimated and ground-truth center—was nearly halved. In Sequence 1, the error dropped from 5.7 pixels to 2.6 pixels.
- Overlap Metrics: The adapted method achieved a 71% spatial overlap with ground truth in Sequence 1, compared to just 45% for the standard method.
These results confirm that the sphere is the "natural" domain for catadioptric vision. By moving the tracking logic to the sphere, the complex distortions of the mirror effectively disappear, allowing the tracker to perceive the world as it is, rather than as it is warped.
Conclusion
This paper demonstrates that adapting visual tracking to the spherical domain is a highly effective strategy for catadioptric sensors. The combination of spherical coordinates, geodesic distances, and multi-part histograms provides a robust framework for handling extreme geometric distortion and challenging lighting. Future work will explore the integration of shape and texture features to reduce reliance on chromatic information and extend the framework to multi-object tracking in panoramic environments.
Summary updated