TL;DR
QuadricSLAM introduces a novel object-oriented SLAM framework that uses 2D object bounding boxes from multiple views to simultaneously estimate camera poses and 3D ellipsoids (dual quadrics) representing objects in the scene. By deriving a robust geometric error formulation that correctly accounts for partially visible or truncated objects, the system creates compact, semantically meaningful maps while implicitly performing loop closures based on object observations.
What Problem Does This Paper Address?
Traditional Simultaneous Localization and Mapping (SLAM) systems typically represent the world using low-level geometric primitives like points, lines, or planes. While highly effective for tracking, these maps lack semantic meaning—a robot sees a cloud of points rather than a chair, a table, or a monitor.
To bridge this gap, "semantic SLAM" has emerged. However, existing approaches often require prior knowledge of exact 3D CAD models or perform dense reconstructions where semantics are painted on post-hoc, which can be computationally heavy and brittle. Previous attempts to use quadrics (3D surfaces like ellipsoids) as compact object landmarks suffered from limitations: they either assumed simplistic orthographic cameras, required complex pre-processing steps like ellipse fitting, or relied on algebraic error metrics that failed catastrophically when objects were partially occluded or cut off by the image boundary.
QuadricSLAM addresses these issues by allowing a standard perspective camera to use raw 2D object detections (bounding boxes) to directly constrain and estimate 3D object landmarks within a full SLAM optimization loop.
Methods
The core innovation of QuadricSLAM lies in how it models object landmarks and calculates the error between what the robot expects to see and what it actually sees.
- Dual Quadric Parametrization: The authors represent 3D objects as constrained dual quadrics ($Q^*$), specifically ellipsoids. An ellipsoid is highly compact, requiring only 9 degrees of freedom to capture an object's 3D position, orientation, and size along its three semi-axes.
- The Bounding Box Sensor Model: When a 3D ellipsoid is projected onto a 2D image plane, it forms a conic (an ellipse). A naive sensor model would draw a bounding box around this ellipse and truncate it at the image borders. However, if the object is partially off-screen, this introduces massive errors. QuadricSLAM introduces a novel geometric sensor model ($\beta$) that analytically computes the exact intersection points between the conic and the image boundaries, ensuring a mathematically rigorous bounding box prediction even for truncated objects.
- Factor Graph Optimization: The system is formulated as a factor graph using the GTSAM library. It jointly optimizes the robot's trajectory (odometry factors) and the quadric parameters (landmark factors) by minimizing a geometric error term—the pixel distance between the predicted bounding boxes and the observed bounding boxes from an object detector like YOLOv3.
Datasets
The authors evaluated QuadricSLAM using both simulated and real-world data:
- Synthetic Dataset (UnrealCV): To evaluate mapping accuracy against absolute ground truth, the authors generated 50 trajectories across 10 indoor scenes using an Unreal Engine plugin. They corrupted these trajectories with noise across 5 different seeds, resulting in a large-scale benchmark of 250 simulation trials.
- TUM RGB-D Dataset: For real-world validation, they used four standard sequences (
fr1_desk,fr1_desk2,fr2_desk, andfr3_office). Object detections were provided by YOLOv3, and data association (knowing which bounding box belonged to which physical object) was manually annotated to isolate the SLAM performance.
Key Insights and Ideas
- Objects as Loop Closures: By treating objects as landmarks, the robot can recognize a previously seen table or monitor from a completely different viewpoint, implicitly triggering a loop closure that corrects accumulated odometry drift.
- Geometric > Algebraic Error: The paper highlights that standard algebraic errors (commonly used in projective geometry for ease of optimization) shrink the estimated quadrics when objects are only partially visible. Switching to a geometric bounding-box error makes the optimization robust to the messy realities of robot navigation where objects frequently enter and leave the frame.
- Compact Semantics: Instead of storing millions of points, the map stores a handful of ellipsoids containing semantic labels, positions, and scales, providing a lightweight yet rich representation for high-level robotic tasks.
Key Results
Simulation
In the synthetic trials, QuadricSLAM demonstrated massive improvements over raw odometry and standard initialization techniques:
- Trajectory Accuracy: Achieved a 65.2% improvement in Absolute Trajectory Error (ATE) compared to raw odometry.
- Landmark Quality: Improved landmark position estimation by 70.4%, shape estimation by 26.7%, and overall landmark quality by 30.6% compared to the initial SVD (Singular Value Decomposition) linear solution.
Real-World (TUM RGB-D)
On real-world sequences, QuadricSLAM successfully bounded odometry drift, significantly outperforming raw visual odometry (fovis). While it slightly underperformed compared to a highly optimized point-based feature SLAM (ORB-SLAM2), the authors showed that by filtering out highly volatile object detections (rejecting bounding boxes with high variance), they could achieve competitive trajectory errors (e.g., 2.39 cm on fr1_desk2).
Limitations and Future Work
While a major step forward for object-oriented mapping, the authors note several limitations:
- Data Association: The system currently assumes that data association is solved (i.e., the robot knows which bounding box corresponds to which landmark). Integrating an automated data association algorithm is a crucial next step.
- Occlusions and Non-Gaussian Noise: Bounding boxes from deep learning detectors often suffer from non-Gaussian noise due to partial occlusions, which can cause the estimated quadrics to "shrink."
- Initialization Behind the Camera: The mathematical formulation of quadric projection allows objects to initialize behind the camera plane in rare, low-viewing-angle scenarios. Future work will focus on enforcing front-of-camera constraints during initialization.
Summary updated