A Simulation-Based Evaluation Framework for Interactive AI Systems and Its Application

Maeda F. Hanafi, Yannis Katsis, Martín Santillán Cooper, Yunyao LiAAAI 2022 · DOI

Introduction

Evaluating Interactive AI (IAI) systems presents a unique challenge: performance is intrinsically tied to unpredictable human behavior. In traditional AI, models are evaluated against static datasets. In IAI, the user is an active participant—for instance, highlighting text examples to "teach" an extraction model. If a user provides suboptimal examples, the AI's performance suffers, making it difficult for developers to distinguish between algorithmic flaws and poor user input.

Currently, developers often rely on qualitative, ad-hoc user testing. This approach is slow and prone to the "whack-a-mole" problem, where fixing one bug inadvertently introduces others. The InteractEva framework addresses this by introducing a simulation-based approach to quantitatively track performance and debug IAI systems at scale.

The InteractEva Framework

InteractEva is designed to provide data-driven guidance for IAI development by decoupling the evaluation from live human testing. The framework consists of two primary components: a simulation backend and an interactive frontend.

The InteractEva framework architecture, featuring a User Simulation Engine that bridges the evaluated IAI system with user models and use cases.

Simulation Backend

At the core of the framework is the User Simulation Engine. Instead of a human tester, this engine interacts with the AI using User Models—scripts that simulate various behaviors, such as a "lazy" user providing minimal examples versus a "thorough" user. These models are run against diverse Use Cases (datasets with established ground truth) to generate thousands of simulated interaction sessions.

Interactive Frontend

The frontend is a comprehensive dashboard that aggregates the data from these simulated runs. It allows developers to visualize distributions of precision and recall, compare different versions of the AI backend, and perform deep-dive analyses into specific sessions to identify exactly where a model failed.

Experimental Application and Insights

The framework was validated using a commercial NLP pattern induction system within IBM Watson Discovery. The evaluation covered several complex use cases, including technical ISO numbers, COVID-19 statistics, and varied measurement units.

One significant discovery facilitated by InteractEva was the "More is Less" Paradox. Simulation data revealed that in early iterations, providing more positive examples sometimes caused performance to drop. The algorithm struggled with the increased variability of the input—a subtle deficiency that likely would have remained hidden during limited manual testing.

The error analysis dashboard provides an aggregate view of use cases, allowing developers to filter runs by performance metrics and inspect learned rules.

Furthermore, the dashboard enabled Automated Error Analysis. By filtering for low-scoring runs, developers found the system was learning overly generic "wildcard" rules rather than specific patterns. This insight led to the integration of regular expressions into the learner, significantly improving robustness.

Key Results and Impact

The integration of InteractEva into the industrial development lifecycle yielded several benefits:

  • Quantitative Benchmarking: Developers moved from anecdotal evidence to concrete metrics, such as comparing F1 scores across 100 runs per use case.
  • Regression Tracking: Automated simulations allowed the team to track performance across 14 software iterations, ensuring that new features did not degrade performance on existing use cases.
  • Evidence-Based UX Guidelines: By conducting "what-if" studies with different user models, the team developed data-backed recommendations for users regarding the optimal number of seed examples required for high-accuracy results.

Limitations and Future Work

InteractEva is primarily algorithm-centered, focusing on the robustness of the underlying logic rather than the user interface. While the User Models serve as a proxy for human behavior, they do not yet account for human factors like fatigue or cognitive biases. Future research will focus on developing more sophisticated user models and adapting the framework for other IAI domains, such as image labeling and interactive recommendation systems.

Summary updated

BibTeX
@inproceedings{hanafi2022simulation,
  author = {Maeda F. Hanafi and Yannis Katsis and Martín Santillán Cooper and Yunyao Li},
  title = {{A Simulation-Based Evaluation Framework for Interactive AI Systems and Its Application}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2022},
  doi = {10.1609/aaai.v36i11.21541},
}

Loading related content...

Figure