OSRE: Object-to-Spot Rotation Estimation for Bike Parking Assessment

Saghir Alfasly, Zaid Al-Huda, Saifullahi Aminu Bello, Ahmed Elazab, Jian Lu, Chen Xu

TL;DR

OSRE (Object-to-Spot Rotation Estimation) is a new deep learning framework designed to assess how well a bicycle is parked by estimating its rotation in two axes relative to its parking spot. By training on a massive new synthetic dataset (SynthBRSet) and using clever image-smoothing techniques for real-world deployment, the researchers have enabled AI to detect not just where a bike is, but whether it is upright, fallen, or misaligned.

What Problem Does This Paper Address?

The rise of dockless bike-sharing systems has led to a major urban management headache: "parking chaos." While current AI models are great at finding a bike in an image (object detection), they are remarkably bad at telling you if that bike is "well-parked."

To a computer, a bike lying flat on the sidewalk and a bike standing upright in a rack look like the same "class" of object. To assess parking quality, a model needs to understand rotation—specifically, rotation relative to the parking area (the "spot"), not just the camera. The two main hurdles have been:

  1. Data Scarcity: Manually measuring and labeling the exact 3D angles of thousands of parked bikes is a logistical nightmare.
  2. Contextual Rotation: Standard detectors ignore the relationship between an object and its environment (e.g., the ground plane).

Methods

The authors propose the Object-to-Spot Rotation Estimator (OSRE). Here’s how it works:

  • Architecture: Built on top of the state-of-the-art YOLOv7 detector, OSRE adds a specialized regression head. While standard YOLO predicts a bounding box and a class, OSRE also regresses two rotation angles: $y$ (representing leaning or falling left/right) and $z$ (representing the bike's orientation/direction).
  • Spot-Awareness: Unlike "camera-agnostic" models that only care about how an object looks to the lens, OSRE is "spot-aware." It learns to implicitly derive the parking area's orientation from the visual context of the image to calculate the bike's relative angle.
  • Sim-to-Real Bridge: Because the model is trained entirely on "clean" synthetic data, it can struggle with the messy textures of the real world. To fix this, the authors use image smoothing/filtering (like 2D convolution and low-pass filters) during deployment. This effectively "blurs" the real-world details to better match the synthetic features the model learned during training.

Datasets

The backbone of this research is the newly created SynthBRSet (Synthetic Bike Rotation Dataset):

  • Scale: 139,581 bike objects across 20,284 images.
  • Generation: Created using the Blender 3D engine, allowing for perfect, automated ground-truth labels for rotation.
  • Diversity: Includes variations in lighting, weather, backgrounds (grass, pavement), and "clutter" like pedestrians and vehicles.
  • Real-World Validation: To prove it works, they also collected and manually annotated a small real-world dataset of 175 bike objects using a physical protractor for ground-truth measurements.

Key Insights and Ideas

  • The "Smoothing" Paradox: Usually, researchers try to make synthetic data look more "real." Here, the authors found success doing the opposite: making real images look more "synthetic" via smoothing. This significantly boosted performance on real-world data, reducing rotation error by nearly 40%.
  • Implicit Context: The model doesn't need the parking spot to be explicitly outlined or labeled. It learns the "semantic context" of what a parking area looks like, allowing it to estimate rotation relative to a ground plane that it "understands" but hasn't been told to find.
  • Two-Axis Necessity: By splitting rotation into $y$ (fallen vs. standing) and $z$ (aligned vs. crooked), the model provides a complete "parking score" that can trigger specific alerts (e.g., "Bike #402 has fallen over" vs. "Bike #402 is blocking the path").

Key Results

  • Detection Accuracy: The model maintained high detection performance, achieving 93.8 mAP (Mean Average Precision) on synthetic data.
  • Rotation Precision: On real-world images, the best-performing version (using a YOLOv7-X backbone) achieved a Mean Squared Error (MSE) of 0.028 for rotation estimation.
  • Backbone Superiority: YOLOv7-X consistently outperformed older versions like YOLOv3 or YOLOv4, proving that the latest architectural improvements in object detection directly benefit complex tasks like rotation regression.

Limitations and Future Work

While a major step forward, the authors note a few areas for improvement:

  • Viewpoint Bias: The current model is optimized for cameras looking from the top or behind the parking area. Front-facing views are less represented.
  • Dataset Size: The real-world test set is relatively small (175 objects). Expanding this would help further validate the model's robustness.
  • Beyond Bounding Boxes: The authors suggest that moving away from bounding boxes toward "object-centric point regression" might lead to even more accurate angle predictions in the future.

Check out the project: The data and code are available at https://github.com/saghiralfasly/OSRE-Project.

Summary updated

Abstract

Current deep models excel in object detection for classification and localization. However, precise object rotation estimation within the visual context of an input image remains underexplored due to the lack of object datasets with rotation annotations. This paper addresses these challenges by tackling rotation estimation for parked bikes with respect to their parking area. Firstly, 3D graphics were leveraged to build a camera-agnostic well-annotated Synthetic Bike Rotation Dataset (SynthBRSet). Subsequently, an object-to-spot rotation estimator (OSRE) is introduced by extending object detection to regress bike rotations in two axes. As the proposed model trained purely on synthetic data, image smoothing techniques adopted during deployment on real-world images. The proposed OSRE has undergone evaluation on both synthetic and real-world data, showing promising results. Our data and code are available at https://saghiralfasly.github.io/OSRE-Project/.
BibTeX
@article{alfasly2024osre,
  author = {Saghir Alfasly and Zaid Al-Huda and Saifullahi Aminu Bello and Ahmed Elazab and Jian Lu and Chen Xu},
  title = {{OSRE: Object-to-Spot Rotation Estimation for Bike Parking Assessment}},
  journal = {IEEE Trans. ITS},
  year = {2024},
  doi = {10.1109/tits.2023.3330786},
}

Loading related content...

Figure