Latent Event-Predictive Encodings through Counterfactual Regularization

Dania Humaidan, Sebastian Otte, Christian Gumbsch, Charley M. Wu, Martin V. Butz

Abstract: Human cognition relies on the ability to segment continuous sensory streams into discrete, manageable events. This paper introduces SUGAR (SUrprise-GAted Recurrent neural network), a model designed to learn stable latent event encodings. By incorporating a novel Counterfactual Regularization (CFR) loss, SUGAR learns to anticipate transitions and maintain consistent internal representations, overcoming common limitations in traditional recurrent neural networks such as high prediction error at boundaries and unstable latent codes.

Introduction

Intelligent systems face a constant "firehose" of continuous sensory data. To make sense of this stream, humans naturally segment information into discrete events—for example, distinguishing the act of "walking to the kitchen" from "making tea." While Recurrent Neural Networks (RNNs) are often used to model sequential data, they frequently struggle with this segmentation. They tend to be reactive, experiencing massive spikes in prediction error (surprise) only after a transition occurs, and they often fail to maintain stable internal representations of a specific context.

The SUGAR Architecture

To address these challenges, the authors propose the SUGAR architecture. It is composed of four specialized LSTM-based layers designed to manage information flow and contextual transitions:

  • Event Processing Layer: Responsible for predicting the immediate next step in a sequence.
  • Event Anticipation Layer: Provides top-down contextual guidance to the processing layer.
  • Event Switching Layer: Acts as a gate, determining when to update the processing layer with new contextual information.
  • Event Boundary Layer: Learns to predict the timing of upcoming transitions based on external cues.
Comparison of SUGAR variants: reactive (a), anticipatory (b), and anticipatory with counterfactual regularization (c).

The core innovation is Counterfactual Regularization (CFR). Standard gating mechanisms can be unstable, opening and closing in response to noise. CFR adds a penalty to the loss function by evaluating a "what if" scenario: If the gate had remained closed, would the prediction have been significantly worse? If not, the model is penalized for opening the gate. This inductive bias forces the model to find the most compact and stable latent representations possible.

Experimental Setup

The model was evaluated using a Hierarchical Sequence Prediction Task. In this task, the system must predict letters in sequences generated by three distinct "problems" or hidden graph structures. The task switches between these problems at random intervals, requiring the model to adapt its internal state to the new context.

The Hierarchical Sequence Prediction Task structure, showing the three distinct graph problems (A-B, B-C, and A-B-C-B).

Inputs to the model include the current letter, a "Contextual Information" (CI) signal hinting at the next problem, and an "Event Boundary" (EB) signal indicating an imminent switch.

Key Results and Insights

The study compared three variants of the architecture: a reactive version (SUGARa), an anticipatory version (SUGARb), and the full model with CFR (SUGARc). Several key insights emerged:

  • Fluid Transitions: While the reactive SUGARa suffered from significant error spikes at every transition, the anticipatory models (SUGARb and SUGARc) achieved nearly fluid transitions with minimal prediction error.
  • Stability through CFR: Without regularization, latent codes tended to drift. SUGARc, however, produced perfectly stable and consistent codes across multiple occurrences of the same event. This led to higher reliability across different random initializations.
  • Emergent Compositionality: Remarkably, the model learned a semantic map of the events. The latent codes for Problem 3 (which shares components with Problems 1 and 2) naturally positioned themselves between the codes for 1 and 2 in vector space, suggesting emergent compositional understanding.

Conclusion and Future Work

SUGAR demonstrates that combining surprise-gated mechanisms with counterfactual reasoning allows artificial agents to learn structured, stable representations of continuous data. While the current model relies on external boundary signals, future work aims to enable agents to infer these boundaries purely from sensory data and to test the architecture on more complex, high-dimensional streams like video or robotic sensorimotor data.

Summary updated

BibTeX
@inproceedings{humaidan2021latent,
  author = {Dania Humaidan and Sebastian Otte and Christian Gumbsch and Charley M. Wu and Martin V. Butz},
  title = {{Latent Event-Predictive Encodings through Counterfactual Regularization}},
  booktitle = {CogSci},
  year = {2021},
}
Figure