Abstract
Domain adaptation methods for 2D human pose estimation typically require continuous access to the source data during adaptation, which can be challenging due to privacy, memory, or computational constraints. To address this limitation, we focus on the task of source-free domain adaptation for pose estimation, where a source model must adapt to a new target domain using only unlabeled target data. Although recent advances have introduced source-free methods for classification tasks, extending them to the regression task of pose estimation is non-trivial. In this paper, we present Prior-guided Self-training (POST), a pseudo-labeling approach that builds on the popular Mean Teacher framework to compensate for the distribution shift. POST leverages prediction-level and feature-level consistency between a student and teacher model against certain image transformations. In the absence of source data, POST utilizes a human pose prior that regularizes the adaptation process by directing the model to generate more accurate and anatomically plausible pose pseudo-labels. Despite being simple and intuitive, our framework can deliver significant performance gains compared to applying the source model directly to the target data, as demonstrated in our extensive experiments and ablation studies. In fact, our approach achieves comparable performance to recent state-of-the-art methods that use source data for adaptation.
Introduction
Standard Unsupervised Domain Adaptation (UDA) typically requires a model to learn from a new "target" domain while maintaining simultaneous access to the original "source" dataset. While effective, this requirement is often impractical in real-world scenarios. Human pose datasets frequently contain Personally Identifiable Information (PII), making them difficult to share due to privacy regulations. Furthermore, the sheer scale of these datasets—often reaching terabytes of video—poses significant storage and bandwidth challenges.
In response, researchers are shifting toward Source-Free Domain Adaptation (SFDA). While SFDA has seen success in image classification, it remains a significant challenge for Human Pose Estimation (HPE). Because HPE is a regression task requiring the prediction of exact joint coordinates, models are prone to "forgetting" anatomical constraints when exposed to new environments. Without the source data to act as an anchor, a model might begin predicting biologically impossible poses. To solve this, we introduce Prior-guided Self-training (POST).
Methodology: The POST Framework
POST is designed to adapt a pre-trained source model to a target domain using only unlabeled target images. The framework rests on three primary pillars:
1. Mean Teacher Framework
To maintain stability during adaptation, POST employs a Mean Teacher architecture. This involves two versions of the same model: a "Student" and a "Teacher." The Student learns actively from the target data, while the Teacher’s weights are updated as an exponential moving average (EMA) of the Student’s weights. This ensures that the pseudo-labels provided by the Teacher remain stable, preventing the model from deviating too sharply from its initial knowledge base.
2. Dual-Level Consistency
To ensure the model learns robust features rather than just memorizing noise, POST enforces consistency at two levels:
- Prediction-level Consistency: The model is required to produce identical pose predictions for an image even when that image undergoes transformations like rotation, scaling, or blurring.
- Feature-level Consistency: Utilizing the Barlow Twins objective, the framework ensures that the internal representations (features) extracted by the model are informative and non-redundant. This helps the model understand the underlying structure of the human body across different environments.
3. The Human Pose Prior
The core innovation of POST is the integration of a human pose prior. This is a lightweight network trained exclusively on joint coordinates to recognize the "manifold" of plausible human poses. It acts as an anatomical sanity check. If the Student model predicts a pose that is physically impossible—such as an elbow bending in an unnatural direction—the Pose Prior penalizes the prediction. This regularization forces the adaptation process to stay within the bounds of biological reality, effectively replacing the need for source data with the universal laws of human anatomy.
Experimental Evaluation
We evaluated POST across several diverse datasets to test its robustness against distribution shifts:
- SURREAL to Human3.6M: Transitioning from synthetic data to real-world indoor environments.
- LSP (Leeds Sports Pose): Testing the model on athletes in complex, outdoor athletic poses.
- BRIAR: A challenging biometric dataset featuring subjects at extreme distances (up to 500 meters) and captured from UAV (drone) perspectives.
Key Results
Performance was measured using [email protected] (Percentage of Correct Keypoints within a 5% error margin).
- Significant Gains: In the SURREAL → Human3.6M benchmark, the baseline "Source only" model (no adaptation) achieved 67.3%. POST improved this significantly to 76.7%.
- Competitive Performance: Despite having no access to source data, POST achieved results comparable to state-of-the-art UDA methods (like RegDA) that require full source data access.
- Robustness in Extremes: On the BRIAR dataset, POST successfully localized joints at distances of 500 meters, where atmospheric turbulence and low resolution caused the original source model to fail entirely.
Conclusion and Future Work
POST demonstrates that anatomical priors can effectively substitute for source data in domain adaptation for regression tasks. By anchoring the model in the physical reality of human movement, we can achieve high-performance pose estimation while respecting data privacy and reducing computational overhead.
Future research will look toward extending this source-free logic to 3D pose estimation, where depth ambiguity presents new challenges, and exploring multi-person scenarios where occlusions and crowded scenes complicate the self-training process.
Summary updated