The Amazing Race TM: Robot Edition

Jared Sigurd Johansen, Thomas Victor Ilyevsky, Jeffrey Mark SiskindarXiv 2020 · arXiv

Introduction

Researchers at Purdue University have addressed a fundamental challenge in robotics: how can a robot navigate a completely unknown, unmodified office environment without pre-existing maps? Their solution, inspired by "The Amazing Race™," involves a robot that functions like a human visitor—by finding people and asking for directions. This approach moves away from traditional "crutches" like SLAM (Simultaneous Localization and Mapping) and instead relies on social navigation, natural language processing, and common-sense reasoning.

System Architecture and Methods

The system is built upon a Finite State Machine (FSM) that orchestrates five primary behaviors: WANDER, APPROACH_PERSON, HOLD_CONVERSATION, FOLLOW_DIRECTIONS, and NAVIGATE_DOOR.

The Finite State Machine (FSM) architecture showing the sequence of behaviors and failure recovery loops.

A critical feature of this architecture is its robustness through failure recovery. Unlike linear models, the FSM allows the robot to loop back to previous states. For instance, if a person walks away during a conversation or a direction is misunderstood, the robot reverts to the WANDER state to find a new person rather than failing the mission entirely. This design increased the success rate by a factor of 13.3 compared to a non-looping model.

Intersection Detection and Classification

To follow instructions like "take the second left," the robot must translate low-level LiDAR data into high-level topological features. The researchers developed an algorithm to classify intersections (e.g., elbows, T-junctions, or four-way crossings) by analyzing directional sectors around the robot.

LiDAR-based sensing and intersection classification: (a) directional sectors, (b-d) identifying hallway features and intersections.

This qualitative mapping mimics human spatial reasoning. Instead of precise GPS coordinates, the robot builds a graph where nodes represent intersections and edges represent hallways. Once in the target hallway, the robot uses Google OCR and a Line Segment Detector (LSD) to read door tags and find the specific goal.

Experimental Results

The system was evaluated in 52 live trials across three previously unseen buildings at Purdue University (HAMP, KNOY, and ME), involving 13 untrained volunteers.

  • Overall Success: The robot achieved a 76.9% success rate in finding specific rooms.
  • Building Performance: Success varied by layout, with the highest rate (84.2%) in the ME building and the lowest (64.7%) in HAMP.
  • State Reliability: While the WANDER state was nearly 99% successful, the NAVIGATE_DOOR state proved most challenging (40% success) due to OCR fragility and occluded door tags.

Key Insights and Future Work

The study demonstrates that "zero-shot" deployment in novel environments is possible by leveraging human assistance. The robot's ability to use common-sense logic—such as inferring room number trends to skip unnecessary door checks—significantly optimized its search process.

Future improvements will focus on making OCR more robust and expanding the dialogue system to understand non-verbal cues like hand gestures. Additionally, the researchers aim to integrate neural networks for more sophisticated intersection classification in complex architectural layouts.

Summary updated

Abstract

State-of-the-art natural-language-driven autonomous-navigation systems generally lack the ability to operate in real unknown environments without crutches, such as having a map of the environment in advance or requiring a strict syntactic structure for natural-language commands. Practical artificial-intelligent systems should not have to depend on such prior knowledge. To encourage effort towards this goal, we propose The Amazing Race TM: Robot Edition, a new task of finding a room in an unknown and unmodified office environment by following instructions obtained in spoken dialog from an untrained person. We present a solution that treats this challenge as a series of sub-tasks: natural-language interpretation, autonomous navigation, and semantic mapping. The solution consists of a finite-state-machine system design whose states solve these sub-tasks to complete The Amazing Race TM. Our design is deployed on a real robot and its performance is demonstrated in 52 trials on 4 floors of each of 3 different previously unseen buildings with 13 untrained volunteers.
BibTeX
@misc{johansen2020amazing,
  author = {Jared Sigurd Johansen and Thomas Victor Ilyevsky and Jeffrey Mark Siskind},
  title = {{The Amazing Race TM: Robot Edition}},
  year = {2020},
  eprint = {2010.15033},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2010.15033},
}
Figure