Learning Dynamics of Attention: Human Prior for Interpretable Machine Reasoning

Wonjae Kim, Yoonho Lee

Abstract

Without relevant human priors, neural networks may learn uninterpretable features. We propose Dynamics of Attention for Focus Transition (DAFT) as a human prior for machine reasoning. DAFT is a novel method that regularizes attention-based reasoning by modelling it as a continuous dynamical system using neural ordinary differential equations. As a proof of concept, we augment a state-of-the-art visual reasoning model with DAFT. Our experiments reveal that applying DAFT yields similar performance to the original model while using fewer reasoning steps, showing that it implicitly learns to skip unnecessary steps. We also propose a new metric, Total Length of Transition (TLT), which represents the effective reasoning step size by quantifying how much a given model's focus drifts while reasoning about a question. We show that adding DAFT results in lower TLT, demonstrating that our method indeed obeys the human prior towards shorter reasoning paths in addition to producing more interpretable attention maps.

Introduction

In Visual Question Answering (VQA), neural networks often function as "black boxes." While they may achieve high accuracy, their internal reasoning processes—often visualized through attention maps—can be erratic, redundant, or unaligned with human logic. For example, when tasked with finding the color of an object relative to another, a model might scan the entire scene unnecessarily. This lack of a human-like prior for concise, smooth reasoning and the absence of quantitative metrics to measure reasoning efficiency hinder the development of truly interpretable AI.

Methods: Dynamics of Attention for Focus Transition (DAFT)

To address these issues, we introduce DAFT (Dynamics of Attention for Focus Transition). Unlike traditional models that update attention in discrete, independent steps, DAFT treats attention as a state within a continuous dynamical system.

Comparison of discrete attention updates in MAC vs. continuous dynamics in DAFT MAC.

By utilizing Neural Ordinary Differential Equations (Neural ODEs), DAFT models the infinitesimal change of attention over time. This approach forces the model to learn a continuous "trajectory" of focus, mimicking the smooth shifts in human attention. We integrated this method into the MAC (Memory, Attention, and Composition) network, creating "DAFT MAC."

Additionally, we propose the Total Length of Transition (TLT) metric. Grounded in the Minimum Description Length (MDL) principle, TLT quantifies the "distance" a model's focus travels by summing the Jensen-Shannon divergence between consecutive attention maps. A lower TLT indicates a more efficient and interpretable reasoning path.

Results and Discussion

We evaluated DAFT MAC on the CLEVR and GQA benchmarks. A key finding is that DAFT naturally promotes reasoning efficiency. While the original MAC model required 12 discrete steps to reach peak accuracy on CLEVR, DAFT MAC achieved the same state-of-the-art performance (98.9%) in only 4 steps. This suggests that DAFT filters out redundant noise inherent in standard attention mechanisms.

Qualitative comparison of attention maps and reasoning paths between MAC and DAFT MAC, including TLT scores.

As shown in the qualitative analysis, DAFT MAC produces significantly more interpretable attention maps. In specific examples, DAFT reduced the TLT from 5.42 down to 1.50. This numerical reduction corresponds to a shift from a scattered, multi-step focus path to a clean, concise path that aligns directly with the linguistic structure of the question. Furthermore, because DAFT uses ODEs, the reasoning process is continuous, allowing for "infinite resolution" visualizations of the model's thought process at any arbitrary point in time.

Conclusion

DAFT introduces a powerful human prior for machine reasoning, improving both efficiency and interpretability. By modeling attention as a continuous flow, we reduce the number of required reasoning steps by two-thirds while maintaining competitive accuracy. Future work will explore applying the DAFT prior to Natural Language Processing and investigating its impact on model robustness.

Summary updated

Abstract

Without relevant human priors, neural networks may learn uninterpretable features. We propose Dynamics of Attention for Focus Transition (DAFT) as a human prior for machine reasoning. DAFT is a novel method that regularizes attention-based reasoning by modelling it as a continuous dynamical system using neural ordinary differential equations. As a proof of concept, we augment a state-of-the-art visual reasoning model with DAFT. Our experiments reveal that applying DAFT yields similar performance to the original model while using fewer reasoning steps, showing that it implicitly learns to skip unnecessary steps. We also propose a new metric, Total Length of Transition (TLT), which represents the effective reasoning step size by quantifying how much a given model's focus drifts while reasoning about a question. We show that adding DAFT results in lower TLT, demonstrating that our method indeed obeys the human prior towards shorter reasoning paths in addition to producing more interpretable attention maps. Our code is available at https://github.com/kakao/DAFT.
BibTeX
@inproceedings{kim2019learning,
  author = {Wonjae Kim and Yoonho Lee},
  title = {{Learning Dynamics of Attention: Human Prior for Interpretable Machine Reasoning}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2019},
  url = {https://openreview.net/forum?id=E1MM47mEgB1},
}
Figure