TL;DR
This paper introduces a novel computational framework, Cognitive Tomography (CT), to infer individualized internal models of how humans learn complex sequences. By analyzing response times, CT reveals that humans initially rely on a simple "Markovian" inductive bias, which is gradually replaced by a more sophisticated model reflecting the true environmental statistics. This work provides a powerful tool to understand how individual learning strategies and biases shape our perception of the world.
What Problem Does This Paper Address?
Understanding how humans learn and adapt to their environment is a central question in cognitive science. A key concept is the "internal model," a representation of the world's regularities that guides our predictions and actions. While "ideal observer" models provide a normative benchmark for how an optimal agent should learn, they often fail to capture the nuances of human behavior, particularly the influence of prior beliefs or "inductive biases." Humans don't start from a blank slate; they bring pre-existing assumptions that shape how they interpret new information. The challenge is to disentangle these inductive biases from the evidence gathered from the environment, especially when dealing with complex, dynamic, and novel situations where the "true" underlying model is unknown. This paper aims to address this by developing a method to infer these individualized internal models and understand the interplay between inductive biases and environmental evidence during learning.
Methods
The core innovation is the Cognitive Tomography (CT) framework. CT is a non-parametric Bayesian approach that combines probabilistic programming with a generative model of behavior. It aims to reverse-engineer an individual's internal model by analyzing their response times in a task.
Here's a breakdown:
- Internal Model: CT models the participant's internal representation of the environment as an Infinite Hidden Markov Model (iHMM). This allows for arbitrarily complex latent structures and dynamics, without pre-specifying the number of states.
- Response Time Model: A LATER (Linear Ascend to Threshold with Ergodic Rate) model is used to link the internal model's predictions to the observed response times.
- Doubly Bayesian Approach: The framework is "doubly Bayesian" because it accounts for both the uncertainty in the participant's internal model and the uncertainty in the experimenter's model of the participant.
- Inference: A custom sampling method, mixing Hamiltonian Monte Carlo (HMC) and Gibbs sampling, is used to infer the parameters of both the internal model and the response time model.
- Decomposition: CT allows for the decomposition of the inferred internal model into two key components:
- A Markov model component, capturing immediate temporal dependencies between observations. This is hypothesized to represent an inductive bias.
- An Ideal Observer model component, reflecting the learned statistics of the environment.
Datasets
The study used data from an Alternating Serial Reaction Time (ASRT) task. Participants performed this task over ten sessions, with the stimulus sequence changing in later sessions. The data collected consisted of response times for each trial. The paper also describes validation using synthetic datasets generated from known internal models to test the CT framework's recovery capabilities.
Key Insights and Ideas
- Inductive Biases are Structured and Dynamic: The research demonstrates that inductive biases aren't just random noise; they have a specific structure (initially Markovian) and evolve over time. This suggests a principled way humans approach novel learning problems.
- Learning is a Trade-off: Humans don't blindly follow their initial biases or the incoming evidence. Instead, they dynamically trade off reliance on their initial Markovian bias with the information gleaned from the environment (represented by the ideal observer model).
- Individualized Learning Trajectories: The CT framework reveals that these learning trajectories and the strength/persistence of inductive biases vary significantly across individuals, offering a window into different learning strategies.
- CT as a Powerful Tool: The study validates CT as a method capable of inferring complex, dynamic internal models from behavioral data, providing a richer understanding than simpler models.
Key Results
- Early Learning Dominated by Markov Model: Individual learning curves showed that internal models were initially dominated by the Markov model, capturing immediate dependencies.
- Shift Towards Ideal Observer: Over time, the contribution of the ideal observer model (reflecting environmental statistics) increased, suggesting participants learned the true structure of the task.
- CT Outperforms Simpler Models: The CT model consistently outperformed the ideal observer and Markov models in predicting response times, especially in individual trials. For example, on Day 8, CT achieved an average R² of 0.248, significantly outperforming other models.
- Error Prediction: CT also showed superior performance in predicting when errors would occur and biasing responses towards expectations in error trials.
- Model Decomposition: The predictive power of CT could be largely explained by a combination of the Markov and ideal observer model contributions, supporting the idea that these two components capture distinct aspects of the internal model.
Limitations and Future Work
- Assumptions of LATER Model: The study relies on the LATER model for response time prediction, which assumes a specific form for the distribution of response times. While validated on synthetic data, alternative response time models could be explored.
- Computational Cost: Inferring complex, high-dimensional internal models can be computationally intensive.
- Future Work: The authors suggest extending this framework to investigate other types of inductive biases, explore the neural correlates of these inferred internal models, and apply CT to other domains where complex learning and inference are involved. They also mention the potential for using Partially Observed Markov Decision Processes (POMDPs) for more complex scenarios involving action models.
Summary updated