TL;DR
This paper introduces a novel multinomial processing tree (MPT) model to disentangle true attachment preferences from guessing behavior in sentence processing experiments. By analyzing data from English and Turkish, they demonstrate that this MPT approach provides a more nuanced understanding of how people resolve ambiguous sentences than traditional percentage-based analyses.
What Problem Does This Paper Address?
When people read sentences with ambiguities, like where a relative clause ("who had an accident") attaches to a preceding noun phrase, researchers often ask them to indicate their preferred interpretation. However, participants might not always fully process the sentence or pay close attention, leading them to guess their answers. This "guessing" can confound the results, making it difficult to accurately estimate people's true preferences. Previous studies have shown cross-linguistic differences in attachment preferences, but it's unclear how much of this is due to genuine linguistic differences versus methodological artifacts like guessing. This paper aims to address this by developing a method to separate genuine attachment preferences from guessing behavior.
Methods
The core contribution of this paper is the development and application of Multinomial Processing Tree (MPT) models. MPT models are a way to formalize hypotheses about cognitive processes by representing them as a tree of sequential events, where each path through the tree corresponds to a unique combination of cognitive processes and leads to a specific observable response.
The authors propose two MPT models:
- Model 1: This model assumes that participants can either be in an "attentive" state (leading to a correct response based on their interpretation) or an "inattentive" state (leading to a random guess). The probability of being in the attentive state is denoted by 'a', and the probability of guessing 'yes' when inattentive is 'g'. This model assumes that inattentiveness affects all attachment conditions equally.
- Model 2: This model builds upon Model 1 by introducing the possibility that specific attachment processes (N1 or N2) might be more prone to failure (e.g., less likely to be successfully created or recalled). This is captured by parameters
r1andr2, representing the certainty of recollection for N1 and N2 attachments, respectively. This model allows for different error rates for N1 and N2 attachment.
The models were implemented using Bayesian inference with the brms and rstan packages in R.
Datasets
The authors used existing datasets from two experiments:
- English Experiment: Data from Swets et al. (2008), a self-paced reading experiment involving English speakers.
- Turkish Experiment: Data from an unpublished self-paced reading experiment involving Turkish speakers.
Both experiments involved participants reading sentences with ambiguous and unambiguous relative clause attachments and answering polar comprehension questions.
Key Insights and Ideas
- Guessing is a significant confound: The paper strongly argues that guessing behavior is a pervasive issue in psycholinguistic tasks, particularly those involving discrete choice responses, and can significantly distort estimates of cognitive preferences.
- MPT models offer a powerful solution: By formalizing the process of guessing and distinguishing it from genuine interpretation, MPT models provide a way to "de-confound" these factors and obtain more accurate estimates of underlying cognitive parameters.
- Cross-linguistic differences might be exaggerated: The analysis suggests that apparent differences in attachment preferences between English and Turkish might be partly explained by differing processing difficulties (and thus higher guessing rates) for certain attachment types in Turkish.
- Model 2 provides a better fit: The more complex Model 2, which allows for condition-specific error rates, provided a better fit to the experimental data, suggesting that attachment processes are indeed affected differently by error-generating mechanisms.
Key Results
- Model 2 outperforms Model 1: Model comparison using PSIS-LOO-CV indicated that Model 2 had significantly better out-of-sample performance, suggesting that allowing for differential error rates in N1 and N2 attachment processes is crucial.
- English N2 attachment preference holds: In the English experiment, Model 2 confirmed a genuine N2 attachment preference, even after accounting for guessing.
- Turkish N1 attachment preference is likely due to guessing: In the Turkish experiment, what appeared as an N1 attachment preference in raw percentages was largely explained by Model 2 as a consequence of high guessing rates associated with N2 attachment processing difficulties. The parameter
h(probability of N1 interpretation) was not significantly different from chance (0.5) in Turkish, while recall probabilities (r1andr2) showed a large difference, indicating N1 interpretations were much more successfully recalled. - Accuracy varies: Average accuracy in unambiguous conditions was 79% for English and 66.5% for Turkish, highlighting the presence of errors even when the sentence structure was clear.
Limitations and Future Work
- Generalizability of guessing: While the paper focuses on guessing due to inattentiveness, other factors contributing to guessing (e.g., fatigue, distractions) are subsumed under this umbrella term. Further research could explore these specific factors.
- Nature of N2 attachment difficulty in Turkish: The paper speculates on why N2 attachment might be more difficult in Turkish (e.g., due to the pre-nominal relative clause structure and potential interference from discarded N1 structures), but this remains an area for further investigation.
- Parameter estimation uncertainty: While Model 2 fits well, some parameter estimates (particularly
hin Turkish) still had wide credible intervals, indicating ongoing uncertainty. - Future work: The authors suggest that this MPT-based methodology can be applied to other areas of psycholinguistics and cognitive science where discrete choice tasks are used and guessing might be a factor. They also highlight the need for further research into the specific mechanisms that lead to differential processing difficulties in different languages.
Summary updated