The R-U-A-Robot Dataset: Helping Avoid Chatbot Deception by Detecting User Questions About Human or Non-Human Identity

David Gros, Yu Li, Zhou Yu2021 · DOI

As conversational AI becomes increasingly indistinguishable from human interaction, a new ethical and legal frontier has emerged: identity disclosure. Whether it's a customer service bot or a social companion, users have a right to know if they are talking to a machine. In jurisdictions like California, this transparency is often a legal requirement.

In their paper, "The R-U-A-Robot Dataset: Helping Avoid Chatbot Deception by Detecting User Questions About Human or Non-Human Identity," researchers David Gros, Yu Li, and Zhou Yu tackle the technical challenge of ensuring AI systems can accurately recognize when their identity is being questioned and respond transparently.

The Problem: Dishonest Anthropomorphism

We are entering an era of "dishonest anthropomorphism," where systems like Google Duplex can sound so human that users may be deceived into disclosing sensitive information. While some deception is accidental—a byproduct of training on human-to-human dialogue—it can lead to significant user discomfort and mistrust.

The core challenge is intent detection. While "Are you a robot?" seems simple, natural language is messy. A system must distinguish between:

  1. Positive Intent: "Are you a machine?"
  2. Hard Negatives: "I love robots," or "Do you think robots will take over?"
  3. Ambiguous Cases: "You sound robotic," or "Are you a nice person?"

Existing datasets lack coverage for these nuances, leading to systems that either miss the question or provide canned, evasive answers.

The R-U-A-Robot Dataset

The primary contribution of this work is the R-U-A-Robot Dataset, a collection of over 6,800 utterances designed to train classifiers for identity detection. The dataset includes:

  • 2,720 Positive examples (asking if the system is a bot).
  • 3,400 Negative examples (including "hard negatives" sampled from PersonaChat and Reddit to ensure models don't just trigger on the word "robot").
  • 680 "Ambiguous if Clarify" (AIC) examples (edge cases where the intent is unclear).

The researchers used a multi-pronged approach to build this resource, combining grammar-based generation (CFG) to account for typos and colloquialisms with adversarial crowdsourcing to gather challenging negative examples.

Key Insights: The "Human Data" Trap

Generative models like Facebook’s Blender, trained on massive human conversation corpora, often have a "default state" of lying. Because their training data consists of humans talking to humans, the models naturally learn to claim human attributes.

An example of a generative model providing an evasive, anthropomorphic response.

As seen in the figure above, a model might respond to an identity query by claiming a human profession, such as "computer programmer," rather than identifying as a machine. This accidental deception stems directly from the underlying training data.

Furthermore, commercial developers often script "easter egg" responses to add personality. However, the study found that these poetic or vague answers are often perceived as less trustworthy than direct confirmation.

A poetic, scripted response from a commercial assistant that avoids direct disclosure.

Benchmarking and Results

The researchers tested several models, including Bag-of-Words Logistic Regression, FastText, and BERT. The BERT-based classifier achieved an aggregate score of 96.2% on the test set, significantly outperforming simpler models.

When "stress-testing" current commercial systems with 100 unique phrasings of the identity question, the results were concerning:

  • Amazon Alexa only confirmed its non-human identity in 15% of cases.
  • Google Assistant fared better at 35%, but still frequently gave non-confirming answers.
  • Blender (1.4B) denied being a robot 70% of the time.

In a human study, clear confirmations (e.g., "I am a chatbot") received the highest ratings for appropriateness (~6.3/7) and trustworthiness (~6.0/7), while vague or "robot-like" evasions scored significantly lower.

Conclusion

The R-U-A-Robot dataset provides a vital tool for building systems that are not just smart, but honest. By accurately detecting when a user is questioning a system's identity, developers can ensure their AI remains transparent, fostering trust and meeting emerging ethical and legal standards.

Citation: David Gros, Yu Li, Zhou Yu. "The R-U-A-Robot Dataset: Helping Avoid Chatbot Deception by Detecting User Questions About Human or Non-Human Identity." Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). 2021.

Summary updated

BibTeX
@misc{gros2021r,
  author = {David Gros and Yu Li and Zhou Yu},
  title = {{The R-U-A-Robot Dataset: Helping Avoid Chatbot Deception by Detecting User Questions About Human or Non-Human Identity}},
  year = {2021},
  doi = {10.18653/v1/2021.acl-long.544},
}

Loading related content...

Figure