Visualisation and 'Diagnostic Classifiers' Reveal how Recurrent and Recursive Neural Networks Process Hierarchical Structure (Extended Abstract)

Dieuwke Hupkes, Willem H. Zuidema

Introduction

A fundamental question in artificial intelligence is how neural networks process hierarchical and compositional information, such as the nested structures found in natural language. While symbolic models utilize explicit stacks and trees, the internal mechanisms of neural networks often remain a "black box." This research introduces a quantitative method using "diagnostic classifiers" to probe the hidden states of Gated Recurrent Units (GRUs) and Simple Recurrent Networks (SRNs) trained on hierarchical arithmetic tasks.

Methodology: Probing the Hidden State

To isolate the effects of hierarchy, the researchers utilized a synthetic language of nested arithmetic expressions (e.g., ( ( 5 - 7 ) - 6 )). The core innovation is the use of diagnostic classification. Instead of evaluating only the final output, a separate, simple linear classifier—a "probe"—is trained to predict specific intermediate variables from the network's internal hidden states at each timestep.

Architecture of the diagnostic classifier probing the GRU's internal state.

If a probe can accurately reconstruct a variable (like a running total), it demonstrates that the network is actively maintaining that information. This allows researchers to test competing algorithmic hypotheses:

  1. Recursive Strategy: A "proper" method involving a mental stack to store results of sub-expressions.
  2. Cumulative Strategy: A "running total" method that updates the current value and tracks whether the next number should be added or subtracted based on the brackets.

Experimental Setup

The models were trained on arithmetic expressions of varying lengths (L1 to L9, based on the number of numerals). To test true generalization, the models were trained on a subset of lengths (L1, L2, L4, L5, L7) and evaluated on unseen lengths (L3, L6, L8, L9). This ensures the models are learning the underlying mathematical logic rather than simply memorizing patterns.

Results: Uncovering the Cumulative Strategy

The experiments revealed that while SRNs failed to generalize to longer sequences, GRUs demonstrated a robust ability to process unseen, complex structures. Crucially, the diagnostic classifiers showed that GRUs do not implement a classic recursive stack. Instead, they favor a cumulative strategy.

Performance of diagnostic classifiers in predicting the result and mode under different strategy hypotheses.

As shown in Figure 3, the diagnostic classifier for the cumulative strategy achieved a significantly lower mean squared error and higher correlation (Pearson’s $r = 0.95$) with the network's behavior compared to the recursive strategy ($r = 0.52$). Furthermore, the networks tracked the "mode" (the current operation context) with over 93% accuracy even for the longest sequences (L9).

Discussion: Smooth Integration vs. Discrete Logic

A notable finding was that the internal representations in the neural networks changed smoothly over time, rather than in the discrete jumps characteristic of symbolic logic. This suggests that the networks develop a continuous approximation of the underlying mathematical rules. While the cumulative strategy is a "shortcut" compared to full recursion, it proves sufficient for the hierarchical tasks tested and offers a more efficient path for gradient-based learning.

Conclusion and Future Work

By "opening the black box" with diagnostic classifiers, this research provides a rigorous framework for understanding how neural architectures process structure. Future work will explore whether larger models, such as Transformers, develop more sophisticated recursive strategies or continue to rely on refined cumulative approximations when faced with the complexities of natural language.

Summary updated

Abstract

In this paper, we investigate how recurrent neural networks can learn and process languages with hierarchical, compositional semantics. To this end, we define the artificial task of processing nested arithmetic expressions, and study whether different types of neural networks can learn to compute their meaning. We find that simple recurrent networks cannot find a generalising solution to this task, but gated recurrent neural networks perform surprisingly well: networks learn to predict the outcome of the arithmetic expressions with high accuracy, although performance deteriorates somewhat with increasing length. We test multiple hypotheses on the information that is encoded and processed by the networks using a method called diagnostic classification. In this method, simple neural classifiers are used to test sequences of predictions about features of the hidden state representations at each time step. Our results indicate that the networks follow a strategy similar to our hypothesised ‘cumulative strategy’, which explains the high accuracy of the network on novel expressions, the generalisation to longer expressions than seen in training, and the mild deterioration with increasing length. This, in turn, shows that diagnostic classifiers can be a useful technique for opening up the black box of neural networks.
BibTeX
@inproceedings{hupkes2018visualisation,
  author = {Dieuwke Hupkes and Willem H. Zuidema},
  title = {{Visualisation and 'Diagnostic Classifiers' Reveal how Recurrent and Recursive Neural Networks Process Hierarchical Structure (Extended Abstract)}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2018},
  doi = {10.24963/ijcai.2018/796},
}
Figure