Abstract
Explainable artificial intelligence (XAI) methods rely on access to model architecture and parameters that is not always feasible for most users, practitioners, and regulators. Inspired by cognitive psychology, we present a case for response times (RTs) as a technique for XAI. RTs are observable without access to the model. Moreover, dynamic inference models performing conditional computation generate variable RTs for visual learning tasks depending on hierarchical representations. We show that MSDNet, a conditional computation model with early-exit architecture, exhibits slower RT for images with more complex features in the ObjectNet test set, as well as the human phenomenon of scene grammar, where object recognition depends on intrascene object-object relationships. These results cast light on MSDNet's feature space without opening the black box and illustrate the promise of RT methods for XAI.
Introduction
The "black-box" nature of deep neural networks remains a significant hurdle for their deployment in high-stakes environments. While existing Explainable AI (XAI) methods like saliency maps or SHAP values provide insights, they typically require "white-box" access to internal parameters. This paper proposes a novel "black-box" XAI approach inspired by cognitive psychology: Response Time (RT) analysis. In cognitive psychology, RT is used to infer the complexity of mental processes by measuring the interval between a stimulus and a behavior. This paper asks whether the same logic can explain how a Convolutional Neural Network (CNN) processes visual information.
Methodology: RT in Early-Exit Networks
To apply RT analysis to CNNs, the authors utilize dynamic inference models, specifically Multi-Scale DenseNet (MSDNet). Unlike static networks that perform the same amount of computation for every input, MSDNet features multiple "early exits." If a model achieves sufficient confidence at an early layer, it terminates processing; otherwise, it continues to deeper, more computationally intensive layers. By measuring the system clock time (RT) and the softmax confidence at these exits, the researchers can treat the CNN like a human subject in a psychology lab, using RT as a direct correlate for hierarchical processing complexity.
Experimental Results
The study evaluates MSDNet across three primary datasets to test different "stressors" on the model's logic: ImageNet (baseline), ObjectNet (non-canonical viewpoints), and SCEGRAM (scenes with semantic or syntactic violations).
ObjectNet vs. ImageNet
The experiments demonstrate that RT effectively captures the difficulty of visual processing. MSDNet processed ImageNet stimuli 31.11% faster than ObjectNet stimuli (27.40 ms vs. 39.77 ms). The non-canonical views in ObjectNet forced the model to rely on deeper, more computationally expensive layers, resulting in longer RTs.
Scene Grammar and Training Effects
Furthermore, the model exhibited sensitivity to "scene grammar." A three-way repeated-measures ANOVA confirmed a significant effect of semantics on RT ($F(1,61) = 4.87, p = .031$). Inconsistent scenes (e.g., a clock in a pile of dishes) were reliably slower to classify, suggesting that the CNN relies on learned object-object relationships. Interestingly, syntactic (physical) violations had less impact, highlighting a difference between current CNN processing and human visual cognition.
Notably, when the model was trained with self-distillation—a technique that makes early layers more efficient—these RT differences vanished. The model became efficient enough to solve even "hard" scenes at the earliest exits, demonstrating how training regimens directly alter the "cognitive" profile of the AI.
Discussion and Conclusion
RT analysis provides a powerful, non-invasive tool for auditing AI logic. By treating the model as a subject and measuring its temporal signatures, researchers can infer the hierarchical complexity of its feature space without needing access to internal weights or gradients. This method democratizes XAI, allowing stakeholders without privileged access to gain meaningful insights into model behavior and identify which types of stimuli a model finds most challenging.
Summary updated