MiniPose: Miniature-Based Real Data Collection for 6D Pose Estimation of Large Industrial Pipe Segments

Hyunjun Kim, Youngjin Seo, Sungwon Seo, Jiyou Shin, Kunwoo Kim, Chanyong Park, Hyungpil MoonarXiv 2026 · arXiv

TL;DR

MiniPose uses 3D-printed miniatures to bridge the "sim-to-real" gap for 6D pose estimation of massive industrial objects. By capturing real sensor noise on easily handled replicas and geometrically rescaling the data, the authors achieve performance comparable to training on difficult-to-collect full-scale real data.

What Problem Does This Paper Address?

In robotics, 6D pose estimation—determining the exact 3D position and orientation of an object—is a fundamental task. While we’ve gotten quite good at this for small, handheld objects, physically large and heavy industrial components (like the massive pipe segments used in shipyards) remain a major challenge.

The primary bottleneck is data. Deep learning models need thousands of labeled images to learn effectively. For a small toy, you can just toss it on a turntable and snap photos. For a 1.5-meter, 10kg steel pipe, repeatedly repositioning it to capture diverse poses is physically exhausting and spatially demanding.

While synthetic data (simulated images) offers a scalable alternative, it suffers from the "sim-to-real gap." Specifically, real-world depth sensors (like Time-of-Flight cameras) produce complex, structured measurement errors that are incredibly difficult to model mathematically in a simulation. If a model only trains on "perfect" synthetic depth, it often fails when faced with "messy" real-world sensor data.

Methods

The authors propose MiniPose, a pipeline that uses scaled-down replicas as a proxy for real-world data collection.

  1. Fabrication: They 3D-print a 50%-scale miniature of the industrial part and coat it with paint that matches the surface color and reflectance of the full-scale object.
  2. Data Capture: They capture RGB-D data of the miniature in diverse poses using a standard sensor (Microsoft Azure Kinect).
  3. Geometric Rescaling: This is the "aha" moment. Based on a calibrated pinhole camera model, if you scale both the object and the camera distance by the same factor ($s$), the perspective in the RGB image remains identical. However, the depth values must be rescaled by $1/s$ to map them back to the full-scale domain.
  4. Training Mix: They combine a large amount of synthetic data (10,000 images) with a small amount of this "real-miniature" data (1,000 images).
Overview of the MiniPose pipeline showing how miniature data is rescaled to match full-scale geometry.

To ensure the method isn't tied to a specific architecture, they evaluated it on two different 6D pose estimators: PVN3D (which uses 3D keypoint voting) and RDPN6D (which uses dense correspondence regression). They also utilized SAM 2 (Segment Anything Model 2) to automatically mask and crop the objects during the data processing phase.

Datasets

The study focused on six representative industrial pipe types: Bend-45, Straight, Tee, Lateral, Offset-45, and Offset-90.

  • Synthetic Data: 10,000 images per pipe type generated using NVIDIA Isaac Sim.
  • Miniature Data: 1,000 images per pipe type (500 for the Straight pipe) captured at 0.5 scale.
  • Full-Scale Real Data: 100 images per pipe type captured in a real industrial environment for evaluation purposes.
The six pipe geometries across synthetic, miniature, and full-scale domains.

Key Insights and Ideas

  • Real Noise Trumps Modeled Noise: Instead of trying to write complex code to simulate how a sensor fails, MiniPose simply captures the actual failure modes of the sensor on a smaller version of the target.
  • The 10% Rule: You don't need a massive amount of real data. Mixing in miniature data—constituting less than 10% of the total training set—was enough to significantly boost real-world performance.
  • Miniatures as Data Tools: Most previous work used miniatures only for validating algorithms (e.g., testing self-driving cars in a "mini-city"). MiniPose reframes the miniature as a high-fidelity data collection tool for full-scale applications.

Key Results

The authors used the ADD-S AUC metric (Area Under the Curve for average distance of model points) to evaluate accuracy.

  • Performance Boost: For the PVN3D estimator, adding miniature data raised the average AUC from 91.9 (synthetic only) to 94.5.
  • Outperforming Baselines: MiniPose outperformed both analytical noise modeling and test-time depth correction (using the Camera Depth Model).
  • Proxy for Reality: Most importantly, the performance was comparable to training on actual full-scale real data. For PVN3D, MiniPose actually performed better than the full-scale training baseline (94.5 vs 93.4), likely because the ease of handling miniatures allowed for much greater pose diversity in the training set.
Qualitative results showing that models trained with MiniPose (blue) align much better with ground truth than those trained on simulation only (red).

Limitations and Future Work

While promising, the authors acknowledge a few hurdles:

  • Sensor Non-linearity: Real sensor noise doesn't always scale linearly with distance. The rescaled miniature depth is a very good approximation, but not a perfect replica of full-scale noise.
  • Scale Variety: The study only tested a 0.5 scale. Future work is needed to see how performance holds up at even smaller scales (e.g., 0.1 scale).
  • Sensor Generalization: The experiments were conducted using a single type of Time-of-Flight camera. Validating the pipeline across different depth sensing technologies (like structured light or active stereo) remains a future task.

Summary updated

Abstract

6-DOF pose estimation of large and heavy industrial objects, such as large piping systems in shipyards and structural components in petrochemical plants, remains relatively underexplored in robotic perception. Repeatedly repositioning large objects is physically and spatially demanding, making it difficult to acquire real training data with sufficient pose diversity. Synthetic data offer a scalable alternative, but faithfully reproducing real RGB-D observations, particularly the structured measurement errors of real depth sensors, remains difficult. We propose MiniPose, a practical pipeline for miniature-based real RGB-D data collection that addresses both the data-acquisition burden and the sim-to-real gap. We fabricate a scaled-down replica and capture RGB-D data of it in diverse poses using a real RGB-D camera. RGB images are used directly, while depth maps are converted to a geometrically consistent representation of the full-scale object via a simple inverse scaling transformation. This allows training to incorporate real appearance and sensor measurement characteristics that are difficult to reproduce using synthetic data alone. To examine whether the benefit depends on a particular network, we evaluate two RGB-D pose estimators with different formulations, PVN3D and RDPN6D. Mixing at most 1,000 miniature images per pipe type, less than 10\% of the training data, with 10,000 synthetic images improves the mean ADD-S AUC across pipe types over synthetic-only training for both estimators. A comparison with training on full-scale real data further suggests that miniature data can serve as a proxy where full-scale collection is difficult, reducing the data-collection burden for large industrial objects.
BibTeX
@misc{kim2026minipose,
  author = {Hyunjun Kim and Youngjin Seo and Sungwon Seo and Jiyou Shin and Kunwoo Kim and Chanyong Park and Hyungpil Moon},
  title = {{MiniPose: Miniature-Based Real Data Collection for 6D Pose Estimation of Large Industrial Pipe Segments}},
  year = {2026},
  eprint = {2611.15066},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.15066},
}
Figure