Advancing research on unconscious priming: When can scientists claim an indirect task advantage?

Sascha Meyen, Iris A. Zerweck, Catarina Amado, Ulrike von Luxburg, Volker H. Franz

TL;DR

A widely used method for inferring unconscious processing from priming experiments is fundamentally flawed. By reanalyzing 15 influential studies using a more appropriate statistical approach, the authors find little to no evidence for the claimed "indirect task advantage," suggesting that much of the existing literature on unconscious priming may need reinterpretation.

What Problem Does This Paper Address?

The field of consciousness research often relies on "unconscious priming" experiments to argue that cognitive functions can operate outside of conscious awareness. The typical setup involves two tasks: a "direct task" where participants try to consciously identify a masked stimulus (performance is usually near chance), and an "indirect task" where the masked stimulus (prime) influences responses to a subsequent target stimulus (e.g., reaction times). The "standard reasoning" claims that if participants perform poorly on the direct task but show a priming effect in the indirect task, it implies that the prime was processed unconsciously to a higher degree than consciously reportable. This apparent "indirect task advantage" (ITA) is then used to support claims of superior unconscious processing. However, the authors argue that this standard reasoning is mathematically flawed because it compares sensitivities inappropriately, leading to a false impression of an ITA even when none exists.

Methods

The authors propose and utilize a more appropriate statistical analysis that directly compares sensitivities (e.g., using Signal Detection Theory's d' metric) in both the direct and indirect tasks. They developed two main approaches:

  1. Trial-by-Trial Analysis: When full data is available, this method directly calculates d' for both tasks and tests for a significant difference. This involves transforming indirect task measures (like reaction times) into binary classifications to make them comparable to the direct task.
  2. Reanalysis of Summary Statistics: For published studies where trial-by-trial data is unavailable, they developed a method to estimate sensitivities and their differences from reported summary statistics (like t-values and means). This method is designed to be liberal, favoring the detection of an ITA to strengthen their critique.

They also conducted extensive simulations to validate their reanalysis method and to demonstrate the flaws of the standard reasoning.

Datasets

The authors reanalyzed data from 15 influential studies on unconscious priming. They also conducted a replication of the behavioral part of a landmark study by Dehaene et al. (1998). No new datasets were introduced.

Key Insights and Ideas

  • The "Standard Reasoning" is Flawed: The core insight is that the common method of inferring an indirect task advantage (ITA) by comparing near-chance performance in a direct task with a priming effect in an indirect task is mathematically unsound. It fails to properly compare sensitivities.
  • Conflating Effects and Sensitivities: The standard reasoning mistakes a clear congruency effect (e.g., faster reaction times) in the indirect task for evidence of good sensitivity, when in reality, such an effect can arise even from very poor underlying sensitivity.
  • The "Benefit-of-the-Doubt" Approach: To make their critique as strong as possible, the authors developed reanalysis methods that are intentionally biased towards finding an ITA. The fact that even these liberal methods often fail to find evidence for an ITA is a powerful argument against the existence of widespread unconscious processing advantages.
  • Methodological Issues Compound the Problem: Beyond the core statistical flaw, the authors highlight other methodological issues in existing studies (like excluding participants with good direct task performance) that further bias results towards finding an ITA.

Key Results

  • Replication of Dehaene et al. (1998): Their replication study found sensitivities in the direct and indirect tasks to be very similar, with no significant difference, thus finding no evidence for an ITA.
  • Reanalysis of 15 Influential Studies: Out of 44 conditions analyzed across 15 studies that claimed ITAs, only 8 showed a significant difference favoring the indirect task. Crucially, 35 conditions showed no significant difference, and one even showed an advantage for the direct task.
  • Lack of Consistent Evidence: Not a single study provided consistent evidence for ITAs across its conditions, despite all claiming them.
  • Impact on Literature: The findings suggest that a large body of literature on unconscious priming may need to be reinterpreted, as the claims of superior unconscious processing are not robustly supported by the data when analyzed appropriately.

Limitations and Future Work

  • Benefit-of-the-Doubt Approach: While strengthening their critique, the authors acknowledge that their reanalysis method is biased towards finding ITAs. They suggest that future research should use more balanced approaches or, ideally, trial-by-trial data.
  • Need for Community Effort: The authors call for a broad reassessment of the unconscious priming literature by the scientific community, applying the appropriate analysis to substantiate or refute existing claims.
  • Differentiating Conditions: They emphasize that the goal should be to identify the specific conditions (stimuli, tasks, setups) under which a reliable ITA might actually exist, rather than assuming its ubiquity.
  • Beyond Behavioral Data: While their primary focus was on behavioral data, they note that their reanalysis method can be applied to neuroimaging data (EEG, fMRI) as well, which often have even higher noise levels.

Summary updated

Abstract

Current literature holds that many cognitive functions can be performed outside consciousness. Evidence for this view comes from unconscious priming. In a typical experiment, visual stimuli are masked such that participants are close to chance performance when directly asked to which of two categories the stimuli belong. This close-to-zero sensitivity is seen as evidence that participants cannot consciously report the category of the masked stimuli. Nevertheless, the category of the masked stimuli can indirectly affect responses to other stimuli (e.g., reaction times or brain activity)-an effect called priming. The priming effect is seen as evidence for a higher sensitivity to the masked stimuli in the indirect responses as compared with the direct responses. Such an apparent difference in sensitivities is taken as evidence that processing occurred unconsciously. But we show that this "standard reasoning of unconscious priming" is flawed: Sensitivities are not properly compared, creating the wrong impression of a difference in sensitivities even if there is none. We describe the appropriate way to determine sensitivities, replicate the behavioral part of a landmark study, develop methods to estimate sensitivities from reported summary statistics of published studies, and use these methods to reanalyze 15 highly influential studies. Results show that the interpretations of many studies need to be changed and that a community effort is required to reassess the vast literature on unconscious priming. This process will allow scientists to learn more about the true boundary conditions of unconscious priming, thereby advancing the scientific understanding of consciousness. (PsycInfo Database Record (c) 2022 APA, all rights reserved).
BibTeX
@article{meyen2021advancing,
  author = {Sascha Meyen and Iris A. Zerweck and Catarina Amado and Ulrike von Luxburg and Volker H. Franz},
  title = {{Advancing research on unconscious priming: When can scientists claim an indirect task advantage?}},
  journal = {Journal of Experimental Psychology General},
  year = {2021},
  doi = {10.1037/xge0001065},
}
Figure