Supervised Training of Dense Object Nets using Optimal Descriptors for Industrial Robotic Applications

Andras Gabor Kupcsik, Markus Spies, Alexander Klein, Marco Todescato, Nicolai Waniek, Philipp Schillinger, Mathias BürgerAAAI 2021 · DOI

Abstract

Dense Object Nets (DONs) provide a robust framework for robotic perception by mapping images to a dense descriptor space. However, standard self-supervised training often fails in industrial environments due to sensor noise on reflective surfaces and visual ambiguities from rotational symmetries. This paper presents a supervised training approach that utilizes 3D CAD models to derive optimal descriptors via Laplacian Eigenmaps. By embedding object geometry directly into the descriptor space and explicitly handling symmetries, the proposed method significantly enhances the reliability of robotic grasping for challenging industrial parts.

Introduction

Dense Object Nets (DONs) represent a significant advancement in robotic vision, allowing robots to map RGB images into a descriptor space where every point on an object's surface is uniquely identified. This mapping is typically learned through self-supervision, where a robot uses depth data to establish pixel correspondences across multiple camera views.

Figure 1: Comparison between an input RGB image and the resulting dense descriptor map for an industrial part.

While effective for many objects, self-supervision struggles in industrial settings. Consumer-grade depth sensors often produce noisy data when encountering small or highly reflective metallic parts. This noise leads to inaccurate pixel correspondences, resulting in "smeared" descriptors that degrade robotic grasping performance. Furthermore, rotational symmetry—common in industrial components—creates visual ambiguities that standard DONs cannot resolve, as they attempt to assign unique descriptors to points that are visually identical.

Methodology

To overcome the limitations of self-supervision, this research proposes a supervised training regime leveraging known 3D mesh models of industrial objects.

Optimal Embedding via Laplacian Eigenmaps

Instead of allowing the network to discover a descriptor space through contrastive learning, the authors mathematically define an optimal space using Laplacian Eigenmaps (LE). By treating the 3D mesh as a manifold, LE embeds vertices into a $D$-dimensional space such that physical proximity on the object is preserved in the descriptor space with minimal distortion.

Figure 3: Mapping of a 3D mesh model (top) to its optimal descriptor space embedding (bottom).

Handling Symmetry

For objects with rotational symmetry, such as shafts or toolcaps, the authors employ Global Intrinsic Symmetry Invariant Functions (GISIFs). By utilizing specific eigenvectors from the Laplacian, they generate descriptors that are identical for symmetric points. This prevents the network from attempting to learn impossible distinctions between visually identical regions, ensuring stable grasping regardless of axial orientation.

Supervised Rendering and Training

With the 3D model and the object's pose known, the ground-truth descriptor space is rendered directly onto the image plane. A ResNet-34 backbone is then trained using a standard $L_2$ loss to match these rendered targets. This supervised approach bypasses the need for high-quality depth maps during training, relying instead on the geometric "truth" of the CAD model.

Experimental Results

The method was evaluated on two challenging industrial parts: a reflective metallic shaft and a small, symmetric toolcap. Data was collected using a Franka Emika Panda robot equipped with an Intel RealSense D435 camera.

Key findings include:

  • Consistency: The supervised approach eliminated the "long-tail" of catastrophic errors common in self-supervised DONs, such as mistaking background pixels for the object.
  • Grasping Reliability: In pick-and-place tasks, the supervised method consistently achieved successful grasps and correct orientations for the toolcap, whereas the self-supervised baseline frequently failed due to descriptor misalignment.
  • Robustness to Noise: By using geometry as the source of truth, the network remained robust even when the physical sensors provided poor depth information on reflective surfaces.

Conclusion

Transitioning from self-supervised to supervised training for Dense Object Nets significantly improves their utility in industrial robotics. By leveraging 3D models and Laplacian Eigenmaps, researchers can create consistent, symmetry-aware descriptor spaces that are robust to sensor noise. Future work will focus on extending this methodology to multi-object scenes and full 6D pose estimation.

Summary updated

BibTeX
@inproceedings{kupcsik2021supervised,
  author = {Andras Gabor Kupcsik and Markus Spies and Alexander Klein and Marco Todescato and Nicolai Waniek and Philipp Schillinger and Mathias Bürger},
  title = {{Supervised Training of Dense Object Nets using Optimal Descriptors for Industrial Robotic Applications}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2021},
  doi = {10.1609/aaai.v35i7.16759},
}
Figure