Abstract
Standard object detection frameworks typically output 2D bounding boxes, which lack the 3D orientation and geometric depth required for advanced robotics and augmented reality applications. This paper introduces an end-to-end deep learning system designed to detect 3D cuboids in single RGB images. By extending the Faster R-CNN architecture to regress eight vertices and employing a novel iterative feature pooling technique, the proposed method achieves state-of-the-art results in class-agnostic 3D localization.
Introduction
While 2D bounding boxes are a staple of computer vision, they provide limited information about an object's spatial configuration. Traditional methods for 3D reconstruction from 2D images often rely on low-level cues like vanishing points or edges, which frequently fail in cluttered real-world environments. This research bridges the gap by treating the "cuboid" as a fundamental geometric primitive. By training a deep network to recognize "cuboid-ness," the system can localize objects like appliances, furniture, and shipping containers regardless of their specific category.
Methodology
The Deep Cuboid Detector is built upon the Faster R-CNN framework, incorporating several key innovations to handle 3D geometry.
Architecture and Multi-Task Regression
The system utilizes a convolutional backbone (e.g., VGG16) for feature extraction. A Region Proposal Network (RPN) identifies candidate regions. Unlike standard detectors that predict four parameters for a 2D box, this model performs multi-task regression to predict 16 values, representing the $(x, y)$ coordinates for all eight vertices of the cuboid.
Iterative Feature Pooling
A core contribution of this work is the iterative refinement of detections. After an initial pass, the network generates a rough estimate of the cuboid's location. The system then "re-pools" features from this refined region and executes the regression again. This iterative process allows the network to focus its attention on the object's boundaries, significantly improving the precision of corner localization.
Experimental Results
The model was evaluated on the SUN Primitive dataset, which contains over 3,500 images of indoor and outdoor scenes with ground-truth 3D annotations.
Performance and Accuracy
The VGG16-based model with iterative pooling achieved a Mean Average Precision (mAP) of 75.47, representing a substantial improvement over classical HOG-based methods (approx. 24.0 mAP). The iterative refinement alone contributed a 7% increase in the Probability of Correct Keypoint (PCK).
Efficiency
The system is designed for practical deployment. Using a lighter VGG-M backbone, the detector achieves a processing speed of 14 frames per second (fps) on a Titan Z GPU, making it viable for near real-time applications in robotics and AR.
Discussion and Conclusion
The findings suggest that deep convolutional neural networks can implicitly learn complex geometric concepts such as perspective and vanishing points directly from data. While the model occasionally faces challenges with viewpoint ambiguity (e.g., distinguishing identical faces of a cube), the over-parameterized approach of predicting all eight vertices allows the network to infer occluded corners based on visible visual evidence. Future work will focus on mobile optimization and the use of synthetic datasets to further enhance generalization.
Summary updated