DIOR: Dataset for Indoor-Outdoor Reidentification - Long Range 3D/2D Skeleton Gait Collection Pipeline, Semi-Automated Gait Keypoint Labeling and Baseline Evaluation Methods

Yuyang Chen, Praveen Raj Masilamani, Bhavin Jawade, Srirangaraj Setlur, Karthik Dantu

Abstract

In recent times, there is an increased interest in the identification and re-identification of people at long distances, such as from rooftop cameras, UAV cameras, street cams, and others. Such recognition needs to go beyond face and use whole-body markers such as gait. However, datasets to train and test such recognition algorithms are not widely prevalent, and fewer are labeled. This paper introduces DIOR -- a framework for data collection, semi-automated annotation, and also provides a dataset with 14 subjects and 1.649 million RGB frames with 3D/2D skeleton gait labels, including 200 thousands frames from a long range camera. Our approach leverages advanced 3D computer vision techniques to attain pixel-level accuracy in indoor settings with motion capture systems. Additionally, for outdoor long-range settings, we remove the dependency on motion capture systems and adopt a low-cost, hybrid 3D computer vision and learning pipeline with only 4 low-cost RGB cameras, successfully achieving precise skeleton labeling on far-away subjects, even when their height is limited to a mere 20-25 pixels within an RGB frame. On publication, we will make our pipeline open for others to use.

Introduction

Gait recognition—identifying individuals by their walking patterns—is a critical biometric for long-range surveillance where facial features are often indiscernible. However, a significant gap exists in current research: datasets are either captured in highly controlled indoor environments with expensive Motion Capture (MoCap) systems or in outdoor settings with low-resolution imagery that lacks accurate ground-truth labels. This "valley of death" is particularly evident in long-range scenarios where subjects may appear as small as 20–25 pixels in height. Standard pose estimation tools like AlphaPose or OpenPose fail completely at these resolutions, making it nearly impossible to train or evaluate gait-based Re-Identification (Re-ID) systems for real-world security or UAV applications.

The DIOR (Dataset for Indoor-Outdoor Reidentification) framework addresses this by introducing a semi-automated pipeline that leverages multi-view geometry to generate high-fidelity 3D and 2D skeleton labels for both indoor and long-range outdoor environments.

Methodology: Bridging the Resolution Gap

The core innovation of DIOR is its two-pronged approach to data labeling, designed to bridge the gap between high-fidelity indoor data and "in-the-wild" outdoor data.

Indoor Pipeline

In controlled settings, the system utilizes a Vicon MoCap system to track reflective markers with millimeter precision. By employing the Perspective-n-Point (PnP) algorithm, the researchers localized RGB cameras within the MoCap coordinate space. This allows for the direct projection of 3D joint labels onto 2D video frames, achieving pixel-level accuracy.

Outdoor Pipeline

Since MoCap is impractical outdoors, DIOR employs a "triangulate-and-reproject" strategy. The setup includes three close-range cameras and one long-range camera positioned 60 meters away.

The DIOR outdoor pipeline uses multi-view geometry to triangulate 3D skeletons from high-resolution views and reproject them onto long-range frames.

As shown in the figure above, the process involves:

  1. Detecting 2D keypoints in high-resolution close-range views using standard pose estimators.
  2. Triangulating these points into a 3D skeleton.
  3. Reprojecting the 3D skeleton onto the long-range camera's view.

To refine these labels, the authors utilized Bundle Adjustment (via the GTSAM library) to correct for minor camera misalignments, reducing reprojection errors significantly and ensuring the ground truth is highly reliable.

The DIOR Dataset

The resulting DIOR dataset is one of the most comprehensive of its kind, featuring 1.649 million RGB frames across 14 subjects, with 200,000 frames specifically from a long-range (60m) perspective.

Visual representation of the DIOR dataset labeling, showing subjects at various scales and the resulting high-precision skeleton labels even at 20px height.

The figure above illustrates the challenge of long-range detection. At a height of only 20 pixels (Row d), traditional AI models see only noise. DIOR provides the ground truth (Rows e and f) necessary to train the next generation of gait recognition models. Geometry acts as a robust "teacher" here, transferring knowledge from high-resolution views to low-resolution views through 3D space. This method is inherently robust to lighting changes and partial occlusions because the labels are generated from the geometry of other cameras, not the pixels of the long-range camera itself.

Results and Benchmarking

The authors evaluated state-of-the-art gait recognition models, including GaitGraph, GaitMixer, and GaitFormer. A key finding was the total failure of off-the-shelf tools (AlphaPose, MLKIT, YOLOv7) on the 60m footage, where the success rate for joint detection was near 0%. This underscores the necessity of the DIOR labeling pipeline.

While GaitMixer showed the most robust performance in long-range scenarios, the overall high error rates across all models indicate that DIOR serves as a challenging and necessary benchmark for the field.

Conclusion

DIOR provides a scalable, low-cost solution for generating high-quality gait labels in difficult environments. By moving beyond the limitations of standard pose estimation through geometric triangulation, this framework enables research into long-range re-identification that was previously impossible. The authors have committed to making the pipeline open-source, providing a blueprint for other researchers to generate high-quality gait labels in challenging environments.

Summary updated

Abstract

In recent times, there is an increased interest in the identification and re-identification of people at long distances, such as from rooftop cameras, UAV cameras, street cams, and others. Such recognition needs to go beyond face and use whole-body markers such as gait. However, datasets to train and test such recognition algorithms are not widely prevalent, and fewer are labeled. This paper introduces DIOR -- a framework for data collection, semi-automated annotation, and also provides a dataset with 14 subjects and 1.649 million RGB frames with 3D/2D skeleton gait labels, including 200 thousands frames from a long range camera. Our approach leverages advanced 3D computer vision techniques to attain pixel-level accuracy in indoor settings with motion capture systems. Additionally, for outdoor long-range settings, we remove the dependency on motion capture systems and adopt a low-cost, hybrid 3D computer vision and learning pipeline with only 4 low-cost RGB cameras, successfully achieving precise skeleton labeling on far-away subjects, even when their height is limited to a mere 20-25 pixels within an RGB frame. On publication, we will make our pipeline open for others to use.
BibTeX
@inproceedings{chen2024dior,
  author = {Yuyang Chen and Praveen Raj Masilamani and Bhavin Jawade and Srirangaraj Setlur and Karthik Dantu},
  title = {{DIOR: Dataset for Indoor-Outdoor Reidentification - Long Range 3D/2D Skeleton Gait Collection Pipeline, Semi-Automated Gait Keypoint Labeling and Baseline Evaluation Methods}},
  booktitle = {IJCB},
  year = {2024},
  doi = {10.1109/ijcb62174.2024.10744452},
}
Figure