ALMANACS: A Simulatability Benchmark for Language Model Explainability

Edmund M Mills, Shiye Su, Stuart Russell, Scott EmmonsSubmission to ICLR 2024 · Rejected

Abstract

How do we measure the efficacy of language model explainability methods? While many explainability methods have been developed, they are typically evaluated on bespoke tasks, preventing an apples-to-apples comparison. To help fill this gap, we present ALMANACS, a language model explainability benchmark. ALMANACS scores explainability methods on simulatability, i.e., how well the explanations improve behavior prediction on new inputs. The ALMANACS scenarios span twelve safety-relevant topics such as ethical reasoning and advanced AI behaviors; they have idiosyncratic premises to invoke model-specific behavior; and they have a train-test distributional shift to encourage faithful explanations. By using another language model to predict behavior based on the explanations, ALMANACS is a fully automated benchmark. We use ALMANACS to evaluate counterfactual, rationalization, and salience-based explanations. Our results are sobering: when averaged across all topics, no explanation method outperforms the explanation-free control. We conclude that despite modest successes in prior work, developing an explanation method that aids simulatability in ALMANACS remains an open challenge.
BibTeX
@misc{mills2023almanacs,
  author = {Edmund M Mills and Shiye Su and Stuart Russell and Scott Emmons},
  title = {{ALMANACS: A Simulatability Benchmark for Language Model Explainability}},
  year = {2023},
  eprint = {2312.12747},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2312.12747},
}
Figure