Video Based Fall Detection Using Human Poses

Ziwei Chen, Yiye Wang, Wankou Yang

Abstract

This paper presents a real-time fall detection system designed for elderly care, leveraging 3D human pose estimation and temporal convolutions. By decoupling fall detection into 2D pose extraction, 3D lifting, and sequence classification using a Fall Detection Network (FDN) with dilated 1D convolutions, the system achieves high accuracy and efficiency. Evaluated on the NTU RGB+D dataset, the model reaches a 99.83% accuracy rate and maintains real-time performance on standard hardware, offering a privacy-preserving and robust alternative to traditional video-based methods.

Introduction

As the global population ages, the need for reliable, non-intrusive fall detection systems has become critical. Traditional solutions often involve wearable sensors, which can be uncomfortable or easily forgotten, or vision-based systems that process raw video. However, raw video analysis is computationally intensive and sensitive to environmental "visual noise" such as lighting changes and cluttered backgrounds. Furthermore, many existing deep learning models analyze short temporal windows, making it difficult to distinguish slow falls from common actions like sitting down. This paper addresses these challenges by proposing a pose-based pipeline that enhances privacy, reduces computational load, and extends the temporal context for more accurate classification.

Methods

The proposed system utilizes a three-stage pipeline to process video data into a fall classification.

  1. 2D Pose Estimation: A Lightweight Pose Network (LPN) extracts 2D joint coordinates from video frames.
  2. 2D-to-3D Lifting: A residual network transforms these 2D coordinates into 3D poses. This step ensures the system is invariant to camera distance and lighting conditions.
  3. Fall Detection Network (FDN): The FDN processes a sequence of 300 frames (approximately 10 seconds) using 1D dilated convolutions.
Overview of the fall detection pipeline, from video input to 3D pose estimation and final classification.

The use of dilated convolutions is a key innovation, allowing the network to achieve a large receptive field—covering the entire 10-second sequence—without the high parameter count or computational cost of standard deep networks.

Detailed architecture of the Fall Detection Network (FDN) featuring residual blocks and dilated convolutions.

Experimental Setup

The model was evaluated using the NTU RGB+D Action Recognition Dataset, focusing on a subset of 44,372 samples. This dataset includes fall events captured from multiple camera angles (-45°, 0°, and +45°), providing a rigorous test for the system's robustness across different viewpoints.

Results and Discussion

The system achieved a state-of-the-art accuracy of 99.83% on the NTU RGB+D dataset. The use of Weighted Cross Entropy Loss was instrumental in handling the class imbalance typical of fall detection datasets, where "not fall" events far outnumber "fall" events. Key findings include:

  • Privacy and Efficiency: By operating on skeletal data rather than raw pixels, the system inherently protects user privacy and remains efficient enough to run at 18 FPS on a standard CPU and 63 FPS on a GPU.
  • Optimal Joint Selection: The researchers found that using a subset of 16 joints (excluding extremities like fingertips) improved accuracy over using the full 25-joint set, likely due to reduced noise in the input signal.
  • Temporal Robustness: The 300-frame input window allows the model to effectively distinguish falls from similar actions like sitting or lying down, which are common sources of false positives in shorter-window models.

Conclusion

The presented 3D pose-based fall detection system offers a high-performance, privacy-conscious solution for elderly monitoring. While the system demonstrates exceptional accuracy in controlled datasets, future work will focus on improving robustness against physical occlusions and testing in diverse, real-world home environments where furniture may partially obscure the subject.

Summary updated

BibTeX
@inproceedings{chen2021video,
  author = {Ziwei Chen and Yiye Wang and Wankou Yang},
  title = {{Video Based Fall Detection Using Human Poses}},
  booktitle = {Big Data (CCF)},
  year = {2021},
  doi = {10.1007/978-981-16-9709-8\_19},
}
Figure