Introduction
In the study of cognitive science and animal learning, a longstanding tension exists between associative learning models and statistical inference theories. Associative models, such as the classic Rescorla-Wagner (RW) algorithm, describe learning as a trial-by-trial update of associative strengths based on prediction errors. While these models are excellent at explaining empirical phenomena like "blocking," they often lack a deep statistical justification. Conversely, causal reasoning theories posit that learners act as intuitive statisticians, estimating the parameters of a generative model using Maximum Likelihood (ML) estimation.
This paper provides a rigorous mathematical bridge between these two paradigms. By augmenting the RW algorithm to include conjunctive causes, the author demonstrates that simple, trial-by-trial learning rules can effectively perform statistical inference for any generative causal model.
The Challenge of Complex Causal Models
Previous research established that the basic RW algorithm is mathematically equivalent to ML estimation for very simple causal scenarios, such as the $\Delta P$ model. However, this equivalence breaks down for more complex, non-linear scenarios like the "noisy-or" model. The central question addressed in this work is whether the RW algorithm can be generalized to perform ML estimation across all generative causal models, rather than just the simplest ones.
Methodology: Augmenting Rescorla-Wagner
To bridge this gap, the author treats the generative model as a sum of basis functions. The critical innovation is augmentation—adding new variables to represent conjunctive causes. Instead of merely tracking individual weights for isolated stimuli (e.g., "Light" and "Tone"), the augmented model includes weights for their combinations (e.g., "Light AND Tone").
The paper introduces two classes of Generalized Rescorla-Wagner (GRW) algorithms to handle this augmented space:
- GRW Class I: A form of stochastic steepest descent (akin to the Widrow-Hoff rule) that updates weights by minimizing the squared error between predicted and actual outcomes.
- GRW Class II: A decoupled update rule where weights are modified only when specific causes are active, mirroring variant rules proposed in earlier literature.
Because this is a theoretical paper, it relies on mathematical proofs rather than empirical datasets, referencing classic experimental paradigms to ground the models in established psychological phenomena.
Key Findings and Theoretical Insights
Through rigorous proofs, the paper establishes several major theoretical results that expand our understanding of associative learning:
- Theorem 1 (ML Equivalence): The paper outlines the sufficient conditions under which a generalized RW algorithm possesses a unique fixed point that exactly matches the ML estimate of a generative model.
- Solving the Noisy-Or Model: By employing a 3-weight augmented RW algorithm (accounting for two individual causes and their conjunction), the model successfully performs ML estimation for the noisy-or model—a feat impossible for the standard 2-weight RW.
- Coordinate Systems and Non-linear RW: When linear RW appears to fail (such as in Cheng's causal power theory), it is often because it estimates parameters in a different "coordinate system." The exact ML parameters can frequently be recovered via a simple non-linear transformation of the RW weights. Furthermore, the author derives a non-linear RW update rule capable of estimating these parameters directly to within arbitrary accuracy.
- Hierarchy of Learning: A GRW designed for a complex, general model will automatically converge to the correct ML estimates even if the underlying data is generated by a simpler model (like $\Delta P$). This suggests a single, sufficiently complex RW mechanism could handle a wide variety of learning environments.
Limitations and Future Directions
While theoretically robust, the augmented framework has practical limitations. The most prominent is the curse of dimensionality: the number of required weights grows exponentially ($2^n$) with the number of causes, making it computationally intractable for environments with many stimuli. Truncating conjunctions at a lower order (e.g., only tracking pairs of causes) is suggested as a practical approximation.
Additionally, the proofs rely on a "genericity assumption," meaning causes cannot be perfectly correlated. While the paper addresses some non-generic cases, a more exhaustive exploration of how the algorithm behaves when causes are highly correlated is needed. Finally, the current proofs focus primarily on the expected value of the weights upon convergence, leaving the analysis of trial-by-trial fluctuations (noise) for future exploration.
Summary updated