Autonomous Upper and Lower Elevator Security Robot Based on Machine Vision

Haoyang Xu, Zheming Yang, Jia Qiu-hongProceedings of the 2018 International Conference on Network, Communication, Computer Engineering (NCCE 2018) 2018 · DOI

Abstract

The security robot realizes the detection of the elevator contour and key by the machine vision algorithm and realizes the control of the elevator door with the manipulator, so as to realize the independent elevator. On the same floor, it achieves the best path and then cruises through path planning and operates the elevator buttons through the manipulator to go up and down the stairs. In case of a critical situation such as a fire, the robot will sound and emit light to alert. It can also achieve positioning through mobile phone APP, start or stop patrols, one-click homing and other functions.

TL;DR

Researchers have developed an autonomous security robot capable of navigating multi-story buildings by independently using elevators. By combining OpenCV-based machine vision for button recognition, a 3-DOF robotic arm for physical interaction, and RFID-based indoor positioning, the robot can patrol complex environments and respond to emergencies like fires without human intervention.

The Challenge of Multi-Story Navigation

While security robots are becoming more common in industrial and commercial settings, most are "floor-bound." Navigating between different levels of a building usually requires expensive infrastructure upgrades or human assistance to operate elevators. This research tackles the challenge of creating a robot that can use existing, unmodified elevator systems. To do this, the robot must solve three distinct problems: accurately identifying elevator buttons from varying distances, precisely aligning a mechanical arm to press those buttons, and maintaining its location within a building where GPS is unavailable.

System Design and Implementation

The authors implemented a multi-layered system design to achieve autonomous floor-to-floor navigation. The architecture integrates several specialized modules to handle positioning, movement, and environmental monitoring.

System architecture of the security robot showing integrated subsystems.

Core Technologies

  • Machine Vision (Template Matching): To find elevator buttons, the system uses OpenCV’s template matching. Because the robot’s distance from the elevator changes, the authors implemented a "step-by-step scaled multiple" approach. The algorithm scales the template image (from 0.1 to 1.0) to find the best match regardless of the robot's perspective.
  • Manipulator Control: The robot features a three-degree-of-freedom (3-DOF) arm. To ensure the finger actually hits the button, they use a PID (Proportional-Integral-Derivative) negative feedback loop. The system calculates the error between the arm's current position in the camera feed and the target button's coordinates, iteratively adjusting until contact is made.
  • Indoor Positioning: Instead of relying on computationally expensive SLAM (Simultaneous Localization and Mapping), the team used RFID markers. Tags are placed at strategic points like corridors, bends, and stairs. As the robot passes a tag, it reads a unique ID to confirm its exact location.
  • Connectivity & Safety: An ESP8266 module provides Wi-Fi connectivity for mobile app control, while LIDAR and ultrasonic sensors handle real-time obstacle avoidance and detect whether elevator doors are open or closed.

Key Insights and Ideas

  • Scaling the Template, Not the Input: A clever optimization mentioned is scaling the small template image rather than the large high-resolution input frame. This saves significant system resources while allowing the robot to recognize buttons as it approaches the elevator.
  • The "Rotate 180°" Logic: The paper outlines a specific behavioral sequence for elevator use: detect door -> enter -> rotate 180 degrees to face the internal buttons -> identify floor button -> wait for destination -> detect door -> exit. This mimics human behavior and ensures the sensors are always facing the most relevant interface.
  • Hybrid Positioning: By combining "low-tech" RFID tags for global positioning with "high-tech" LIDAR for local obstacle avoidance, the authors created a robust navigation system that is less prone to the "drift" often seen in purely odometry-based robots.

Key Results

The vision system considers a match successful when the normalization rate reaches 0.8 or higher. The paper demonstrates successful recognition and coordinate output in real-world testing.

Machine vision output demonstrating successful elevator button recognition and coordinate calculation.

The robot successfully demonstrated the full cycle of patrol: identifying a need to change floors, navigating to the elevator, operating the external and internal buttons via the PID-controlled arm, and exiting at the correct destination. Additionally, the integration of smoke and fire sensors allows the robot to act as a mobile alarm system, using light and sound to alert occupants during a crisis.

Limitations and Future Work

  • Environment Specificity: The current template matching system is highly dependent on the specific elevator model it was trained on. If the robot were moved to a building with different button styles, new templates would likely be required.
  • RFID Dependency: The navigation relies on pre-installed RFID tags. While effective, this means the robot cannot truly "explore" a completely unknown building without prior setup.
  • Mechanical Complexity: While the 3-DOF arm is sufficient for buttons, the paper does not detail how the robot might handle more complex interactions, such as heavy fire doors or security gates that require pulling rather than pushing.

Future work could involve integrating more generalized deep learning models for object detection to make the vision system more adaptable to different elevator designs.

Summary updated

Abstract

The security robot realizes the detection of the elevator contour and key by the machine vision algorithm and realizes the control of the elevator door with the manipulator, so as to realize the independent elevator. On the same floor, it achieves the best path and then cruises through path planning and operates the elevator buttons through the manipulator to go up and down the stairs. In case of a critical situation such as a fire, the robot will sound and emit light to alert. It can also achieve positioning through mobile phone APP, start or stop patrols, one-click homing and other functions.
BibTeX
@inproceedings{xu2018autonomous,
  author = {Haoyang Xu and Zheming Yang and Jia Qiu-hong},
  title = {{Autonomous Upper and Lower Elevator Security Robot Based on Machine Vision}},
  booktitle = {Proceedings of the 2018 International Conference on Network, Communication, Computer Engineering (NCCE 2018)},
  year = {2018},
  doi = {10.2991/ncce-18.2018.15},
}
Figure