A Semi-Autonomous Interactive Robot

Brian Schlesinger, Michael Mensch, Christopher Rindosh, Joe Votta, Yunfeng WangAAAI 2006

Introduction

TARO (A Semi-Autonomous Interactive Robot) represents a significant 2006-era exploration into the integration of high-level logic-based programming with physical social cues. Developed to improve the quality of human-robot interaction (HRI), the project sought to move beyond simple task execution toward a more natural, "utility-based" agent. By bridging a Prolog-based reasoning engine with a Visual Basic interface and a humanoid upper body, the researchers created a system capable of speech recognition, emotional facial expression, and physical gestures such as handshaking.

The Problem: The Non-Verbal Communication Gap

A central challenge in HRI is that effective communication involves far more than processing text or speech. The researchers grounded their work in the "Mehrabian model," which posits that only 7% of communication effectiveness is derived from literal words, while the remaining 93% is attributed to body language, facial expressions, and vocal tone.

Most robotic systems of the era focused heavily on the 7% (the verbal logic), often resulting in interactions that felt mechanical or disjointed. The TARO project aimed to address the 93% gap by integrating physical and visual social cues into a unified architecture, creating a more holistic interactive experience.

Methods and System Architecture

The project utilized a hybrid software and hardware architecture designed to balance complex logic with real-time physical control.

Software Stack

The intelligence of TARO was split between two primary environments:

  • Prolog: This logic-based language was used for high-level reasoning and Natural Language Processing (NLP). The logic was modularized into three distinct areas: Movement, Conversation, and General Logic. These modules were compiled into Dynamic Link Libraries (DLLs) to allow for efficient execution.
  • Visual Basic: This served as the "real-world interface." It acted as the primary host application, calling the Prolog DLLs while managing the graphical user interface (GUI), speech synthesis, and low-level motor control.

Speech Recognition and Contextual Filtering

To overcome the high error rates associated with open-ended dictation in the mid-2000s, the team implemented a modified command-based system using the Microsoft Speech SDK. This system utilized structured word combinations and context-aware filters. By restricting the robot’s vocabulary based on its current state or environment, the researchers drastically increased recognition accuracy and ensured more coherent dialogue flows.

Physical Systems

The hardware was designed to be both functional and cost-effective:

  • Mobile Platform: An aluminum frame (20" x 36") powered by a differential drive system consisting of two 12VDC motors and caster wheels.
  • Upper Body: Constructed from PVC tubing, the upper body featured a monitor that displayed a graphical face. This face mirrored the robot's internal "emotional" state and utilized lip-syncing algorithms to match the text-to-speech output.
  • Robotic Arms: Two arms were dedicated to social gestures. The left arm was designed for waving, while the right arm utilized a feedback loop to detect contact, allowing the robot to simulate a responsive handshake.

Key Insights: Creating the "Illusion" of Interaction

A primary insight from the TARO project was the importance of the "illusion of a speaking entity." The researchers recognized that human users respond more positively when they receive visual feedback that correlates with auditory output. By synchronizing the graphical face with the speech engine, the robot achieved a higher level of perceived social presence.

Furthermore, the use of modular logic ensured system stability. By separating movement and conversation logic into concurrent modules, the team prevented a failure in one domain (such as a misunderstood command) from crashing the entire system. This "state-aware" recognition allowed the robot to maintain the thread of a conversation even within the limitations of its rule-based programming.

Results and Performance

The TARO project successfully demonstrated several key milestones in semi-autonomous robotics:

  1. Effective Integration: The researchers proved that Prolog and Visual Basic could be merged via DLLs to create a responsive, logic-driven AI agent.
  2. High-Accuracy Speech: Moving from open dictation to a constrained, command-based system resulted in significantly higher accuracy rates for understanding and responding to human prompts.
  3. Social Gesture Execution: The robot successfully performed physical interactions, including waving and handshaking. The handshake mechanism was particularly notable for using motor signal changes to mimic human-like responsiveness.

Limitations and Future Work

Despite its successes, TARO remained a "semi-autonomous" system. Its reliance on a command-based speech system, while accurate, limited the robot to a predefined set of word combinations, lacking the flexibility of true natural language understanding. Additionally, the robot required significant environmental constraints or human oversight to function effectively.

Future work for the TARO platform involves evolving the agent toward greater intelligence by expanding the Prolog logic modules and refining the physical responsiveness of the humanoid components to allow for more nuanced social interactions.

Summary updated

BibTeX
@inproceedings{schlesinger2006semi,
  author = {Brian Schlesinger and Michael Mensch and Christopher Rindosh and Joe Votta and Yunfeng Wang},
  title = {{A Semi-Autonomous Interactive Robot}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2006},
}

Loading related content...

Figure