Beyond Human Detection: A Benchmark for Detecting Common Human Posture

Yongxin Li, You Wu, Xiaoting Chen, Han Chen, Depeng Kong, Haihua Tang, Shuiwang Li

TL;DR

This paper introduces a new task called Common Human Posture Detection (CHPD) that goes beyond simply locating humans in images to also identifying their posture (e.g., standing, sitting, bending). To support this, they release the Common Human Postures (CHP) dataset, a benchmark of 5250 images with posture annotations. They also develop and evaluate two baseline detectors, CHP-YOLOF and CHP-YOLOX, demonstrating the feasibility of this new task.

What Problem Does This Paper Address?

Traditional human detection focuses solely on bounding boxes around people. While useful, this is insufficient for many advanced applications like human-computer interaction, healthcare, and autonomous driving. These applications require a deeper understanding of human behavior and state, which includes their posture. Existing human pose estimation methods, which pinpoint individual joints, can be overly complex and struggle with occlusions or low-texture images. This paper bridges this gap by proposing a task that is more informative than simple detection but less demanding than full pose estimation, aiming to provide a practical intermediate step for richer scene understanding.

Methods

The core of this work is the introduction of the Common Human Posture Detection (CHPD) task. This task involves not only detecting humans but also classifying their posture into one of seven common categories: bending, lying, going, running, sitting, squatting, and standing.

To achieve this, the authors adapt existing state-of-the-art object detection architectures. They build upon two identity-preserved human posture detectors, IPH-YOLOF and IPH-YOLOX, which themselves are based on YOLO variants. The key modification is the addition of an extra classification head to predict the common human posture.

For CHP-YOLOF, they use a ResNet50 backbone and incorporate a novel CBAME Module (a combination of ECA and CBAM attention modules) into the posture prediction head to improve feature refinement.

For CHP-YOLOX, they replace the original backbone with InternImage, a more powerful vision transformer, and utilize a PANet neck. Both models are trained using a combined loss function that includes classification, regression, objectness, and posture prediction losses.

Datasets

The primary contribution in terms of data is the introduction of the Common Human Postures (CHP) dataset.

  • Size: 5250 human images.
  • Annotations: Bounding boxes for human detection and labels for seven common human postures.
  • Diversity: Images are collected from various scenes, including public and private spaces, with variations in distance, perspective, and lighting conditions.
  • Data Split: An 8:2 ratio for training and testing (4200 images for training, 1050 for testing).
  • Relation to Existing Datasets: The collection and annotation process follows principles from PASCAL VOC and COCO datasets. The dataset aims to provide richer information than traditional human detection datasets.

Key Insights and Ideas

  • Bridging the Gap: The paper successfully argues for a new task (CHPD) that sits between traditional human detection and detailed human pose estimation, offering a more practical and informative approach for many applications.
  • Posture as Crucial Information: Identifying common human postures provides valuable semantic information for scene understanding, enabling more sophisticated downstream tasks.
  • Challenges in Posture Detection: The study highlights that detecting common human postures is inherently more challenging than just detecting humans, with significant differences in performance metrics. This is attributed to factors like intra-class variation, occlusion, and ambiguity in certain poses (e.g., bending vs. standing).
  • Attention Mechanisms Matter: The ablation studies demonstrate that incorporating attention modules like ECA and CBAME significantly improves posture detection performance.
  • Backbone Architecture is Key: For CHP-YOLOX, using advanced backbones like InternImage leads to substantial performance gains.

Key Results

  • Baseline Performance: Both CHP-YOLOF and CHP-YOLOX demonstrate effective detection of common human postures on the CHP dataset.
  • Quantitative Metrics:
    • CHP-YOLOF achieved an mAPcp of 0.531, while CHP-YOLOX achieved 0.516 at IoU 0.75.
  • The paper reports that detecting common human postures is significantly harder than detecting human categories, with mAPp being considerably lower than mAPc.
  • Performance varies by posture: 'going', 'running', and 'sitting' achieve higher mAPcp (above 50%), while 'bending', 'lying', and 'squatting' are lower (above 40%), and 'standing' is the lowest (below 40%).
  • Qualitative Evaluation: Visualizations show that the detectors perform well in clear, non-occluded scenarios but struggle with complex backgrounds, occlusion, and subtle pose variations.

Limitations and Future Work

  • Performance Gap: The authors acknowledge that the current performance of their proposed architectures is "still not satisfactory."
  • Task Conflict: They note that simultaneously localizing humans and estimating their postures can be challenging, potentially leading to conflicting objectives.
  • Future Directions:
    • Investigate better techniques to reduce the conflict between human detection and posture estimation.
  • Improve the accuracy of detectors, particularly for challenging poses like 'standing' and 'bending'.
  • Explore more advanced network architectures and attention mechanisms.
  • Further analyze and address the specific challenges associated with different postures.

Summary updated

Abstract

Human detection is the task of locating all instances of human beings present in an image, which has a wide range of applications across various fields, including search and rescue, surveillance, and autonomous driving. The rapid advancement of computer vision and deep learning technologies has brought significant improvements in human detection. However, for more advanced applications like healthcare, human–computer interaction, and scene understanding, it is crucial to obtain information beyond just the localization of humans. These applications require a deeper understanding of human behavior and state to enable effective and safe interactions with humans and the environment. This study presents a comprehensive benchmark, the Common Human Postures (CHP) dataset, aimed at promoting a more informative and more encouraging task beyond mere human detection. The benchmark dataset comprises a diverse collection of images, featuring individuals in different environments, clothing, and occlusions, performing a wide range of postures and activities. The benchmark aims to enhance research in this challenging task by designing novel and precise methods specifically for it. The CHP dataset consists of 5250 human images collected from different scenes, annotated with bounding boxes for seven common human poses. Using this well-annotated dataset, we have developed two baseline detectors, namely CHP-YOLOF and CHP-YOLOX, building upon two identity-preserved human posture detectors: IPH-YOLOF and IPH-YOLOX. We evaluate the performance of these baseline detectors through extensive experiments. The results demonstrate that these baseline detectors effectively detect human postures on the CHP dataset. By releasing the CHP dataset, we aim to facilitate further research on human pose estimation and to attract more researchers to focus on this challenging task.
BibTeX
@inproceedings{li2023beyond,
  author = {Yongxin Li and You Wu and Xiaoting Chen and Han Chen and Depeng Kong and Haihua Tang and Shuiwang Li},
  title = {{Beyond Human Detection: A Benchmark for Detecting Common Human Posture}},
  booktitle = {Sensors},
  year = {2023},
  url = {https://openreview.net/forum?id=LytYihTguL},
}
Figure