STL: Surprisingly Tricky Logic (for System Validation)

Ho Chit Siu, Kevin Leahy, Makai Mann

Abstract

Formal methods, particularly Signal Temporal Logic (STL), are frequently touted as "human-interpretable" alternatives to black-box models in autonomous system validation. However, this claim is rarely subjected to empirical testing. This paper presents a study involving 62 participants—ranging from novices to experts—tasked with validating robot motion plans using STL in a "Capture the Flag" gridworld. The results reveal a significant gap between the perceived and actual interpretability of formal specifications: participants achieved a mean accuracy of only 45%, frequently falling victim to affirmation bias. These findings suggest that formal logic, in its current forms, may not provide the safety guarantees often assumed by the research community.

Introduction

In the development of autonomous systems, ensuring safety and reliability is paramount. The formal methods community has long advocated for the use of logic-based specifications, such as Signal Temporal Logic (STL) and Linear Temporal Logic (LTL), to define and verify robot behavior. The core argument for these methods is their "interpretability"—the idea that a mathematical formula is inherently more transparent and auditable by a human operator than a deep neural network.

Despite the prevalence of this claim, a literature review of 25 recent papers asserting the interpretability of formal logic found that zero actually performed human experiments to validate this assumption. This paper addresses this gap by investigating whether humans can accurately predict if a given STL specification will keep a robot safe in a dynamic environment.

Experimental Design

The researchers conducted a human-subject experiment using a custom-designed "Capture the Flag" gridworld. Sixty-two participants were asked to evaluate 60 unique validation cases. In each case, a "Blue Team" agent had to satisfy both safety requirements (avoiding red agents) and liveness requirements (capturing the flag and returning home).

To understand how the presentation of logic affects human understanding, the study tested three different formats:

  1. Raw STL: The standard mathematical formula.
  2. Text-based: The formula translated into indented, English-like blocks.
  3. Tree-based: The formula represented as a hierarchical decision tree.

Participants were asked to determine if a specification was "Valid," "Invalid (No Plan Possible)," or "Invalid (Blue Loss)." The study also incorporated data from a previous study on LTL to determine if observed error patterns were consistent across different temporal logics.

Key Findings

The Interpretability Gap

The most striking result was the low overall performance. The mean validation accuracy across all participants was only 45% ± 20%. Given the three-choice nature of the task, this performance is only marginally better than random chance in several scenarios. This suggests that simply presenting a formal specification does not guarantee that a human can correctly interpret the resulting system behavior.

Affirmation Bias

The study identified a pervasive "affirmation bias" among participants. Subjects were significantly more accurate at identifying valid specifications than invalid ones. When presented with a flawed specification—one that contained a loophole allowing for a catastrophic failure—participants consistently failed to spot the error. They tended to assume the logic was correct simply because it was presented in a formal, structured manner.

The Expertise and Format Paradox

While familiarity with formal methods and higher education levels were positive predictors of success, they were not a panacea. Experts were slightly more accurate than novices but exhibited significantly higher levels of confidence in their answers, even when those answers were incorrect. This "expert overconfidence" suggests that specialized training in writing logic does not necessarily translate to proficiency in auditing it.

Furthermore, while participants expressed a strong preference for the text-based English translations, finding them "easier" to read, their actual validation accuracy did not significantly improve compared to those viewing raw mathematical formulas. This indicates that the difficulty of temporal logic is rooted in the underlying cognitive complexity of the reasoning required, rather than the surface-level syntax.

Discussion and Future Work

The results of this study challenge the status quo in formal methods for robotics. If humans cannot reliably validate specifications, the safety benefits of using "interpretable" logic are severely diminished. The authors suggest several paths forward to bridge this gap:

  • Beyond Syntax: Improving interpretability requires more than just translating math into English. We need "ergonomic logic" representations designed to mitigate known cognitive biases like affirmation bias.
  • Counter-example Generation: Validation tools should move beyond static displays of formulas. Providing humans with visual counter-examples—simulations showing how a specification might fail—could help operators identify loopholes more effectively.
  • Auditing-Focused Training: Current education in formal methods focuses on the synthesis of logic. There is a clear need for training programs focused on the audit and deconstruction of specifications, teaching users how to actively look for ways a system might "cheat" or fail under a given set of rules.

In conclusion, while STL and LTL remain powerful tools for system design, their utility for human-in-the-loop validation is currently limited by human cognitive constraints. Future research must treat interpretability as an empirical claim to be tested rather than an inherent property of mathematical logic.

Summary updated

Abstract

Much of the recent work developing formal methods techniques to specify or learn the behavior of autonomous systems is predicated on a belief that formal specifications are interpretable and useful for humans when checking systems. Though frequently asserted, this assumption is rarely tested. We performed a human experiment $(\mathbf{N}=62)$ with a mix of people who were and were not familiar with formal methods beforehand, asking them to validate whether a set of signal temporal logic (STL) constraints would keep an agent out of harm and allow it to complete a task in a gridworld capture-the-ftag setting. Validation accuracy was 45% $\pm$ 20% (mean $\pm$ standard deviation). The ground-truth validity of a specification, subjects' familiarity with formal methods, and subjects' level of education were found to be significant factors in determining validation correctness. Participants exhibited an affirmation bias, causing significantly increased accuracy on valid specifications, but significantly decreased accuracy on invalid specifications. Additionally, participants, particularly those familiar with formal methods, tended to be overconfident in their answers, and be similarly confident regardless of actual correctness. Our data do not support the belief that formal specifications are inherently human-interpretable to a meaningful degree for system validation. We recommend ergonomic improvements to data presentation and validation training, which should be tested before claims of interpretability make their way back into the formal methods literature.
BibTeX
@inproceedings{siu2023stl,
  author = {Ho Chit Siu and Kevin Leahy and Makai Mann},
  title = {{STL: Surprisingly Tricky Logic (for System Validation)}},
  booktitle = {IEEE/RSJ International Conference on Intelligent Robots and Systems},
  year = {2023},
  url = {https://openreview.net/forum?id=MCh483cN3w},
}
Figure