Abstract
Dense Object Nets (DONs) provide a robust framework for robotic perception by mapping images to a dense descriptor space. However, standard self-supervised training often fails in industrial environments due to sensor noise on reflective surfaces and visual ambiguities from rotational symmetries. This paper presents a supervised training approach that utilizes 3D CAD models to derive optimal descriptors via Laplacian Eigenmaps. By embedding object geometry directly into the descriptor space and explicitly handling symmetries, the proposed method significantly enhances the reliability of robotic grasping for challenging industrial parts.
Introduction
Dense Object Nets (DONs) represent a significant advancement in robotic vision, allowing robots to map RGB images into a descriptor space where every point on an object's surface is uniquely identified. This mapping is typically learned through self-supervision, where a robot uses depth data to establish pixel correspondences across multiple camera views.
While effective for many objects, self-supervision struggles in industrial settings. Consumer-grade depth sensors often produce noisy data when encountering small or highly reflective metallic parts. This noise leads to inaccurate pixel correspondences, resulting in "smeared" descriptors that degrade robotic grasping performance. Furthermore, rotational symmetry—common in industrial components—creates visual ambiguities that standard DONs cannot resolve, as they attempt to assign unique descriptors to points that are visually identical.
Methodology
To overcome the limitations of self-supervision, this research proposes a supervised training regime leveraging known 3D mesh models of industrial objects.
Optimal Embedding via Laplacian Eigenmaps
Instead of allowing the network to discover a descriptor space through contrastive learning, the authors mathematically define an optimal space using Laplacian Eigenmaps (LE). By treating the 3D mesh as a manifold, LE embeds vertices into a $D$-dimensional space such that physical proximity on the object is preserved in the descriptor space with minimal distortion.
Handling Symmetry
For objects with rotational symmetry, such as shafts or toolcaps, the authors employ Global Intrinsic Symmetry Invariant Functions (GISIFs). By utilizing specific eigenvectors from the Laplacian, they generate descriptors that are identical for symmetric points. This prevents the network from attempting to learn impossible distinctions between visually identical regions, ensuring stable grasping regardless of axial orientation.
Supervised Rendering and Training
With the 3D model and the object's pose known, the ground-truth descriptor space is rendered directly onto the image plane. A ResNet-34 backbone is then trained using a standard $L_2$ loss to match these rendered targets. This supervised approach bypasses the need for high-quality depth maps during training, relying instead on the geometric "truth" of the CAD model.
Experimental Results
The method was evaluated on two challenging industrial parts: a reflective metallic shaft and a small, symmetric toolcap. Data was collected using a Franka Emika Panda robot equipped with an Intel RealSense D435 camera.
Key findings include:
- Consistency: The supervised approach eliminated the "long-tail" of catastrophic errors common in self-supervised DONs, such as mistaking background pixels for the object.
- Grasping Reliability: In pick-and-place tasks, the supervised method consistently achieved successful grasps and correct orientations for the toolcap, whereas the self-supervised baseline frequently failed due to descriptor misalignment.
- Robustness to Noise: By using geometry as the source of truth, the network remained robust even when the physical sensors provided poor depth information on reflective surfaces.
Conclusion
Transitioning from self-supervised to supervised training for Dense Object Nets significantly improves their utility in industrial robotics. By leveraging 3D models and Laplacian Eigenmaps, researchers can create consistent, symmetry-aware descriptor spaces that are robust to sensor noise. Future work will focus on extending this methodology to multi-object scenes and full 6D pose estimation.
Summary updated