Shape-Biased Texture Agnostic Representations for Improved Textureless and Metallic Object Detection and 6D Pose Estimation

Peter Hönig, Stefan Thalhammer, Jean-Baptiste Weibel, Matthias Hirschmanner, Markus VinczeWACV 2025

TL;DR

This paper introduces a novel "texture randomization" technique to train deep learning models for object detection and 6D pose estimation. By randomly applying diverse textures to synthetic training data, the models learn to rely on object shape rather than surface appearance, significantly improving performance on challenging textureless and metallic objects and enhancing robustness to image perturbations.

What Problem Does This Paper Address?

Object detection and 6D pose estimation are crucial for robotics and scene understanding. However, current deep learning models, particularly Convolutional Neural Networks (CNNs), exhibit a "texture bias." This means they tend to learn from surface textures, which works well for most objects but fails dramatically for textureless and metallic objects. These objects lack distinct visual cues, and their appearance can drastically change with varying illumination, making them difficult to detect and localize accurately. Existing approaches often rely on extensive synthetic data, but this can still fall short due to the complex interplay of light and surface. This paper aims to overcome this limitation by forcing models to learn shape-based representations, making them agnostic to surface texture.

Methods

The core contribution is a method for inducing a "shape bias" during training. Instead of relying on consistent textures in synthetic data, the authors propose texture randomization. This involves:

  • UV-Mapping Textures: They treat texture application as a UV-mapping process, where 2D textures are projected onto 3D object surfaces.
  • Random Sampling: During data rendering, textures are uniformly sampled from a large database of available textures (1226 in their experiments). This ensures that the model never sees the same texture consistently for a given object across different training samples.
  • Negligible Overhead: This randomization is integrated into existing rendering engines with minimal computational cost for data rendering and network training.

This approach is contrasted with style transfer methods that aim to achieve shape bias but are more computationally intensive. The authors hypothesize that this texture randomization will force CNNs to encode object shape into their feature representations, leading to better generalization for textureless and metallic objects.

Datasets

The paper utilizes the following datasets for training and evaluation:

  • TLESS: A dataset containing 30 highly symmetrical, textureless industrial objects.
  • ITODD: A dataset featuring 28 metallic industrial objects.
  • BOP (Benchmark for 6D Object Pose Estimation): The test sets from the BOP challenge are used for evaluation. The training data is generated using the standard configuration of BlenderProc, a tool for rendering synthetic data.

Key Insights and Ideas

  • Texture Bias is Real and Detrimental: The paper confirms that CNNs have a strong bias towards learning from textures, which hinders performance on objects lacking them.
  • Shape Bias is Key for Textureless/Metallic Objects: By forcing models to learn shape-based representations, performance on these challenging object categories can be significantly improved.
  • Texture Randomization is Efficient: The proposed method offers a computationally inexpensive way to induce shape bias, making it practical for real-world applications.
  • Generalization Across Tasks: The shape bias learned through texture randomization benefits not only classification but also object detection and 6D pose estimation.
  • Robustness Gains: Models trained with shape bias show improved robustness to image perturbations like noise and illumination changes.

Key Results

  • Object Classification: The UV-mapping approach for shape bias induction improved top-1 accuracy by 15.01% and top-5 accuracy by 6.81% over style transfer methods on the TLESS dataset.
  • Object Detection:
    • For Faster R-CNN and RetinaNet on TLESS, shape bias improved mAP by over 200% compared to texture bias.
  • On ITODD, shape bias consistently improved mAP for YOLOx, YOLOv8, and YOLOv10, with improvements ranging from 6.03% to 60.31% (default) and 27.47% to 46.20% (tuned).
  • For textureless objects (TLESS), shape bias without augmentations performed nearly as well as tuned augmentations with texture bias.
  • 6D Pose Estimation:
    • Shape-biased GDR-Net showed a 3.38% AR improvement over texture-biased GDR-Net.
  • Integrating shape-biased detectors further boosted GDR-Net AR by up to 3.61% and Pix2Pose AR by up to 7.39%.
  • On ITODD, shape bias significantly improved Pix2Pose AR by up to 18.60%.
  • Robustness: Shape bias improved robustness to Gaussian and Shot image noise, maintaining high accuracy even with significant noise levels.

Limitations and Future Work

  • Mesh Origin: The paper notes that while CAD models provide more accurate geometry, reconstructed objects with texture bias still benefit from shape bias induction. Future work could explore combining accurate geometry with shape bias more effectively.
  • Online Data Augmentation: While shape bias generally helps, achieving optimal performance with object detection often requires tuning online data augmentation parameters.
  • Vision Transformers and Diffusion Models: The study did not explore the impact of shape bias on these newer architectures.
  • ITODD Ground Truth: The ITODD test set lacks ground truth annotations, limiting extensive evaluation. Future work aims to provide more comprehensive evaluations for metallic objects.
  • Adversarial Learning: The authors suggest exploring adversarial learning for shape bias induction to further enhance detection and pose estimation accuracy and robustness.

Summary updated

BibTeX
@inproceedings{honig2025shape,
  author = {Peter Hönig and Stefan Thalhammer and Jean-Baptiste Weibel and Matthias Hirschmanner and Markus Vincze},
  title = {{Shape-Biased Texture Agnostic Representations for Improved Textureless and Metallic Object Detection and 6D Pose Estimation}},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision},
  year = {2025},
}
Figure