Abstract
While there are several widely used object detection datasets, current computer vision algorithms are still limited in conventional images. Such images narrow our vision in a restricted region. On the other hand, 360° images provide a thorough sight. In this paper, our goal is to provide a standard dataset to facilitate the vision and machine learning communities in the 360° domain. To facilitate the research, we present a real-world 360° panoramic object detection dataset, 360-Indoor, which is a new benchmark for visual object detection and class recognition in 360° indoor images. It is achieved by gathering images of complex indoor scenes containing common objects and the intensive annotated bounding field-of-view. In addition, 360-Indoor has several distinct properties: (1) the largest category number (37 labels in total); (2) the most complete annotations on average (27 bounding boxes per image). The selected 37 objects are all common in indoor scenes. With around 3k images and 90k labels in total, 360-Indoor achieves the largest dataset for detection in 360° images. In the end, extensive experiments on the state-of-the-art methods for both classification and detection are provided.
Introduction
Conventional computer vision research primarily focuses on images with a narrow field of view. However, the rise of 360° cameras in robotics, virtual reality, and real estate has created a demand for models capable of processing panoramic data. The primary obstacle is geometric distortion; 360° images are typically represented via equirectangular projection, which causes significant warping near the poles. Standard models trained on datasets like MSCOCO struggle with these distortions.
The 360-Indoor dataset addresses this gap by providing a large-scale, diverse, and real-world benchmark specifically for the 360° domain.
The 360-Indoor Dataset
The 360-Indoor dataset is the largest of its kind, comprising 3,335 images and 89,148 annotated objects. It features 37 object categories—such as chairs, lights, windows, and appliances—grouped into five super-categories.
A defining characteristic of the dataset is its annotation density, averaging 27 bounding boxes per image. Images were sourced from platforms like Flickr and Kuula, ensuring a wide variety of real-world indoor environments.
Methodology
To accommodate spherical geometry, the researchers utilized Bounding Field-of-Views (BFoVs) instead of standard 2D bounding boxes. A BFoV is defined by latitude/longitude coordinates ($\phi, \theta$) and field-of-view dimensions ($h, w$). This approach accounts for "circular padding," where objects may wrap around the image edges.
The study benchmarked several state-of-the-art architectures:
- YOLOv3
- Faster R-CNN
- Feature Pyramid Network (FPN)
- SphereNet: A specialized architecture using spherical convolutions to handle equirectangular distortion.
Results and Discussion
Experimental results revealed several key insights:
- Top Performance: FPN with a ResNet101 backbone achieved the highest accuracy with a 33.6% mean Average Precision (mAP).
- The Pre-training Paradox: Standard models pre-trained on massive conventional datasets (ImageNet/COCO) outperformed specialized spherical models like SphereNet trained from scratch. This suggests that current spherical architectures lack the "feature wealth" provided by large-scale conventional pre-training.
- Anchor Optimization: For YOLOv3, using k-means clustering to generate custom anchor boxes from the 360-Indoor dataset significantly improved performance compared to default COCO anchors.
- Distortion Robustness: While most objects reside in low-distortion regions ($\pm 30^\circ$ latitude), the dataset's inclusion of objects in high-distortion zones ($\pm 60^\circ$ to $\pm 90^\circ$) provides a rigorous test for model robustness.
Conclusion
360-Indoor establishes a challenging new benchmark for the computer vision community. While current models achieve a peak mAP of 33.6%, there is significant room for improvement. Future research may focus on leveraging global context—utilizing the full 360° view to infer object relationships—and developing large-scale spherical pre-training datasets to further enhance model performance in panoramic environments.
Summary updated