TL;DR
LOGO-CAP bridges the performance gap in bottom-up human pose estimation by refining fast, coarse center-offset predictions using a fully end-to-end framework. Inspired by Daniel Kahneman’s "thinking, fast and slow," it lifts initial keypoint estimates into local and global expansion maps, using convolutional message passing and dynamic convolution to achieve state-of-the-art accuracy at near real-time speeds (over 20 FPS).
What Problem Does This Paper Address?
In 2D multi-person pose estimation, bottom-up approaches detect all body joints in an image and group them into individuals. These generally fall into two categories:
- Limb/Grouping-based methods (e.g., Part Affinity Fields, Associative Embeddings): Accurate, but computationally expensive and reliant on non-differentiable heuristic grouping algorithms.
- Direct center-offset regression (e.g., CenterNet): Conceptually simple and fast, representing poses as star graphs offset from a person center anchor. However, direct offset regression struggles with localization accuracy, particularly for long-range limbs and articulated poses.
Existing solutions often apply complex multi-stage post-processing (such as rescoring networks or greedy matching). LOGO-CAP proposes a fully end-to-end, differentiable framework that fixes offset regression inaccuracies without sacrificing speed.
Methods
The authors structure their pipeline around a "Fast Initializer, Slow Solver" paradigm:
[HRNet Backbone] ──┬──> Center Heatmaps & Offset Fields ──> Fast Initializer (Coarse Poses)
│ │
├──> Feature Maps ──> Local KEMs (11x11) │
│ │ │
│ Conv Message Passing (CMP) │
│ │ │
│ ▼ │
│ Keypoint Attraction Maps (KAMs) │
│ │ │
└──> Heatmaps ──────> Global KEMs (97x97) │
│ (Dynamic Conv) ▼
▼ Refined Offsets
Refined Heatmaps ─────────> Final Pose Output
- Fast Keypointer (Initial Prediction): An HRNet backbone predicts keypoint heatmaps, center heatmaps, and 2D offset displacement fields. A quick peak-selection step generates initial coarse poses.
- Keypoint Expansion Maps (KEMs): Instead of committing to isolated coordinates, the model extracts spatial windows around each keypoint:
- Local KEMs ($11 \times 11$): Sampled via bilinear interpolation from latent feature maps to represent local uncertainty.
- Global KEMs ($97 \times 97$): Sampled across a wider receptive field from predicted heatmaps and modulated by a Gaussian prior.
- Convolutional Message Passing (CMP): Concatenated local KEMs pass through convolutional layers with Attentive Normalization to model structural anatomical constraints across body parts. This outputs Keypoint Attraction Maps (KAMs), directly supervised by an Object Keypoint Similarity (OKS) loss.
- Local-Global Contextual Adaptation: The learned KAMs act as dynamic, pose-sensitive convolutional kernels applied directly to the enlarged Global KEMs.
- Pose Decoding: The final keypoint positions are extracted via a convex combination of the top-2 activations in the adapted heatmaps, weighted by the center confidence score.
Datasets
- MS COCO (Keypoint): Evaluated on the
val-2017(5k images) andtest-dev 2017(20k images) benchmarks; trained on the 65k training split. - OCHuman: A dedicated evaluation dataset for heavily occluded human pose scenarios (4,713 images containing 8,110 heavily overlapping person instances). No new datasets were introduced.
Key Insights and Ideas
- Untapped Potential in Center-Offset Regression: The authors observed that if you take vanilla center-offset predictions and simply perform an oracle search in an $11 \times 11$ window, accuracy leaps from 60.1 to 88.9 AP. The coarse model gets close, but standard regressors lack spatial sub-pixel precision.
- Why Plain Local Heatmaps Fail: Simply predicting a local heatmap per joint doesn't work because no new structural context is introduced. To resolve ambiguity, the network must combine inter-joint structural relationships with global contextual grounding.
- Dynamic Kernels as Adapters: Using the structurally-aware local representations (KAMs) as dynamic convolution filters on global heatmaps enables the network to adaptively "pull" predictions toward the true joint center in a fully differentiable manner.
Key Results
- COCO test-dev 2017: LOGO-CAP achieves 68.2 AP with HRNet-W32 and 70.8 AP with HRNet-W48, outperforming prior bottom-up methods like DEKR (67.3 / 70.0 AP with rescoring; 66.6 / 69.3 AP without) and HigherHRNet (68.4 AP).
- OCHuman Benchmark: Demonstrates strong generalization under dense occlusion, reaching 41.2 Val AP / 40.4 Test AP (HRNet-W48), outperforming both top-down and bottom-up competitors (e.g., DEKR at 38.8 / 38.2 AP).
- Inference Speed: Running on a single TITAN RTX GPU, LOGO-CAP processes frames in 48 ms (~20.7 FPS) with HRNet-W32 and 112 ms (~8.95 FPS) with HRNet-W48—faster than DEKR, CenterNet, and PifPaf.
Limitations and Future Work
- Performance Gap to the Upper Bound: Although LOGO-CAP achieves 70.0 AP on COCO val, an oracle search within the same local window reaches 88.9 AP, indicating room for improved refinement architectures.
- Message Passing Capacity: The CMP module relies on lightweight convolutions. The authors note that exploring attention mechanisms or Vision Transformers could improve structural modeling, though likely at the expense of inference frame rate.
Summary updated