Fish-Eye Camera Video Processing and Trajectory Estimation Using 3D Human Models

K. Kottari, Konstantinos K. Delibasis, Vassilis P. Plagianakos, Ilias MaglogiannisLecture Notes in Computer Science 2014 · DOI · OpenAlex

Abstract

This paper presents a robust method for tracking human movement and identifying postures (sitting vs. standing) using a single, ceiling-mounted fish-eye camera. By matching a simplified 3D human model—constructed from "generalized cylinders"—to distorted 2D video silhouettes, the system accurately estimates 3D trajectories and orientations. The approach handles extreme geometric distortion by projecting the 3D model into the camera's native space, providing stability even when traditional image segmentation fails.

Introduction

Automated human activity recognition is a cornerstone of "smart home" technology, particularly for elderly care and security. However, using a single camera often results in a loss of depth information, making it difficult to determine a person's exact location or activity. Fish-eye cameras are popular for indoor monitoring due to their 360-degree field of view, but they introduce extreme geometric distortion. This research aims to bridge the gap between distorted 2D silhouettes and 3D human movement without requiring expensive multi-camera setups or wearable sensors.

Methodology

The authors propose a "top-down" approach that combines 3D modeling with geometric optimization.

3D Human Modeling via Generalized Cylinders

Instead of using a heavy, high-polygon mesh, the authors employ a mathematical framework called "generalized cylinders" (GC). They simplify a standard 3D human model (~27,000 vertices) into a series of elliptical cross-sections spaced 2cm apart. This creates a computationally light "cloud of points" that can be easily manipulated to simulate different poses, such as bending at the knees or waist.

Original high-polygon 3D human mesh used as the basis for the simplified model.

Fish-Eye Projection and Silhouette Matching

Using a previously developed calibration model, the 3D coordinates of the human model are projected into the distorted 2D space of the fish-eye camera. This ensures matching happens in the camera's native geometry rather than attempting to "undistort" the image, which can lead to data loss at the edges.

The system segments the human figure from the video background and uses the Simplex optimization algorithm to adjust the 3D model’s parameters (position, orientation, and pose). The goal is to maximize the intersection area between the segmented video silhouette and the projected 3D model silhouette.

Simplified point cloud models representing standing (a) and sitting (b) postures, derived from generalized cylinders.

Results and Discussion

The system was evaluated using custom indoor video sequences with manually annotated ground truth for "sitting" and "standing" statuses.

  • Accuracy: The system achieved 79.86% accuracy in one test environment and 92.68% in another, with high sensitivity for detecting sitting postures.
  • Robustness: A key insight is that the model-based approach acts as a stabilizer. If a person is partially occluded or segmentation fails due to lighting changes, the 3D model's trajectory constraints help the system recover the person's position more effectively than 2D-only trackers.
  • Performance: On an Intel i5 laptop using Matlab, the system processes frames in approximately 900 milliseconds, requiring about 30 iterations per frame to converge.

Conclusion

While the current implementation demonstrates high accuracy and robustness, the processing speed remains a limitation for real-time applications. Future work will focus on optimizing the software environment and expanding the model to recognize more complex actions, such as falls, which are critical for assistive living technologies.

Summary updated

BibTeX
@inproceedings{kottari2014fish,
  author = {K. Kottari and Konstantinos K. Delibasis and Vassilis P. Plagianakos and Ilias Maglogiannis},
  title = {{Fish-Eye Camera Video Processing and Trajectory Estimation Using 3D Human Models}},
  booktitle = {Lecture Notes in Computer Science},
  year = {2014},
  doi = {10.1007/978-3-662-44654-6\_38},
}
Figure