OSRE: Object-to-Spot Rotation Estimation for Bike Parking Assessment

Saghir Alfasly, Zaid Al-huda, Saifullah Bello, Ahmed Elazab, Jian Lü, Xu Chen

Abstract

Current deep models provide remarkable object detection in terms of object classification and localization. However, estimating object rotation with respect to other visual objects in the visual context of an input image still lacks deep studies due to the unavailability of object datasets with rotation annotations. This paper tackles these two challenges to solve the rotation estimation of a parked bike with respect to its parking area. First, we leverage the power of 3D graphics to build a camera-agnostic well-annotated Synthetic Bike Rotation Dataset (SynthBRSet). Then, we propose an object-to-spot rotation estimator (OSRE) by extending the object detection task to further regress the bike rotations in two axes. Since our model is purely trained on synthetic data, we adopt image smoothing techniques when deploying it on real-world images. The proposed OSRE is evaluated on synthetic and real-world data providing promising results.

Introduction: The Challenge of "Parking Chaos"

The rise of dockless bike-sharing systems has transformed urban mobility but introduced a significant management challenge: "parking chaos." While modern AI models excel at finding a bicycle in an image (object detection), they typically fail to assess the quality of the parking. To a standard computer vision model, a bicycle lying flat on a sidewalk and one standing upright in a designated rack appear as the same object class.

To effectively grade parking, a model must understand rotation relative to the parking spot, not just the camera lens. This task is historically difficult due to the scarcity of precisely labeled 3D rotation data and the complexity of understanding an object's relationship with its environment, such as the ground plane.

A comparison of detection methods: (a) standard axis-aligned bounding boxes, (b) oriented bounding boxes, and (c) the proposed OSRE method which estimates specific rotation angles (Y and Z) relative to the parking spot context.

Methodology: The OSRE Framework

The authors introduce the Object-to-Spot Rotation Estimator (OSRE), a framework designed to detect not just where a bike is, but its physical orientation.

Architecture and Spot-Awareness

Built upon the YOLOv7 detector, OSRE incorporates a specialized regression head. While standard detectors predict a bounding box and a class, OSRE regresses two critical rotation angles:

  • Y-axis: Represents leaning or falling (left/right).
  • Z-axis: Represents the bike's directional orientation relative to the spot.

Unlike "camera-agnostic" models that only analyze the object's appearance to the lens, OSRE is "spot-aware." It learns to implicitly derive the orientation of the parking area from the visual context, allowing it to calculate the bike's angle relative to the ground plane without requiring the spot itself to be explicitly labeled.

The Sim-to-Real Bridge

Because OSRE is trained on "clean" synthetic data, it can struggle with the noisy textures of real-world environments. To bridge this gap, the researchers utilize image smoothing and filtering (such as 2D convolution and low-pass filters) during deployment. This technique effectively "blurs" real-world details to match the synthetic features the model learned during training, a counter-intuitive approach that significantly improves performance.

Datasets: SynthBRSet

The foundation of this research is the Synthetic Bike Rotation Dataset (SynthBRSet). Created using the Blender 3D engine, it contains 139,581 bike objects across 20,284 images. This synthetic approach allows for perfect, automated ground-truth labels for rotation across diverse lighting, weather, and background conditions. To validate the model, the team also collected a real-world dataset of 175 objects, manually annotated using physical protractors.

Key Results and Insights

  • High Precision: The model achieved a 93.8 mAP (Mean Average Precision) for detection on synthetic data and a Mean Squared Error (MSE) of 0.028 for rotation estimation on real-world images.
  • The Smoothing Paradox: Making real images look more "synthetic" via smoothing reduced rotation error by nearly 40% on real-world data.
  • Backbone Performance: YOLOv7-X consistently outperformed older architectures like YOLOv3 or YOLOv4, demonstrating that modern detection backbones directly enhance complex regression tasks like rotation estimation.

Conclusion and Future Work

OSRE provides a complete "parking score" that can distinguish between a bike that has fallen over and one that is simply misaligned. While the current model is optimized for top-down or rear views, future work will focus on expanding viewpoint diversity and exploring "object-centric point regression" to further refine angular accuracy.

The data and code for this project are available at https://github.com/saghiralfasly/OSRE-Project.

Summary updated

Abstract

Current deep models provide remarkable object detection in terms of object classification and localization. However, estimating object rotation with respect to other visual objects in the visual context of an input image still lacks deep studies due to the unavailability of object datasets with rotation annotations. This paper tackles these two challenges to solve the rotation estimation of a parked bike with respect to its parking area. First, we leverage the power of 3D graphics to build a camera-agnostic well-annotated Synthetic Bike Rotation Dataset (SynthBRSet). Then, we propose an object-to-spot rotation estimator (OSRE) by extending the object detection task to further regress the bike rotations in two axes. Since our model is purely trained on synthetic data, we adopt image smoothing techniques when deploying it on real-world images. The proposed OSRE is evaluated on synthetic and real-world data providing promising results. Our data and code are available at \href{https://github.com/saghiralfasly/OSRE-Project}{https://github.com/saghiralfasly/OSRE-Project}.
BibTeX
@article{alfasly2024osre,
  author = {Saghir Alfasly and Zaid Al-huda and Saifullah Bello and Ahmed Elazab and Jian Lü and Xu Chen},
  title = {{OSRE: Object-to-Spot Rotation Estimation for Bike Parking Assessment}},
  journal = {IEEE Trans. ITS},
  year = {2024},
  doi = {10.1109/tits.2023.3330786},
}
Figure