TL;DR
A widely used method for inferring unconscious processing from priming experiments is fundamentally flawed. By reanalyzing 15 influential studies using a more appropriate statistical approach, the authors find little to no evidence for the claimed "indirect task advantage," suggesting that much of the existing literature on unconscious priming may need reinterpretation.
What Problem Does This Paper Address?
The field of consciousness research often relies on "unconscious priming" experiments to argue that cognitive functions can operate outside of conscious awareness. The typical setup involves two tasks: a "direct task" where participants try to consciously identify a masked stimulus (performance is usually near chance), and an "indirect task" where the masked stimulus (prime) influences responses to a subsequent target stimulus (e.g., reaction times). The "standard reasoning" claims that if participants perform poorly on the direct task but show a priming effect in the indirect task, it implies that the prime was processed unconsciously to a higher degree than consciously reportable. This apparent "indirect task advantage" (ITA) is then used to support claims of superior unconscious processing. However, the authors argue that this standard reasoning is mathematically flawed because it compares sensitivities inappropriately, leading to a false impression of an ITA even when none exists.
Methods
The authors propose and utilize a more appropriate statistical analysis that directly compares sensitivities (e.g., using Signal Detection Theory's d' metric) in both the direct and indirect tasks. They developed two main approaches:
- Trial-by-Trial Analysis: When full data is available, this method directly calculates d' for both tasks and tests for a significant difference. This involves transforming indirect task measures (like reaction times) into binary classifications to make them comparable to the direct task.
- Reanalysis of Summary Statistics: For published studies where trial-by-trial data is unavailable, they developed a method to estimate sensitivities and their differences from reported summary statistics (like t-values and means). This method is designed to be liberal, favoring the detection of an ITA to strengthen their critique.
They also conducted extensive simulations to validate their reanalysis method and to demonstrate the flaws of the standard reasoning.
Datasets
The authors reanalyzed data from 15 influential studies on unconscious priming. They also conducted a replication of the behavioral part of a landmark study by Dehaene et al. (1998). No new datasets were introduced.
Key Insights and Ideas
- The "Standard Reasoning" is Flawed: The core insight is that the common method of inferring an indirect task advantage (ITA) by comparing near-chance performance in a direct task with a priming effect in an indirect task is mathematically unsound. It fails to properly compare sensitivities.
- Conflating Effects and Sensitivities: The standard reasoning mistakes a clear congruency effect (e.g., faster reaction times) in the indirect task for evidence of good sensitivity, when in reality, such an effect can arise even from very poor underlying sensitivity.
- The "Benefit-of-the-Doubt" Approach: To make their critique as strong as possible, the authors developed reanalysis methods that are intentionally biased towards finding an ITA. The fact that even these liberal methods often fail to find evidence for an ITA is a powerful argument against the existence of widespread unconscious processing advantages.
- Methodological Issues Compound the Problem: Beyond the core statistical flaw, the authors highlight other methodological issues in existing studies (like excluding participants with good direct task performance) that further bias results towards finding an ITA.
Key Results
- Replication of Dehaene et al. (1998): Their replication study found sensitivities in the direct and indirect tasks to be very similar, with no significant difference, thus finding no evidence for an ITA.
- Reanalysis of 15 Influential Studies: Out of 44 conditions analyzed across 15 studies that claimed ITAs, only 8 showed a significant difference favoring the indirect task. Crucially, 35 conditions showed no significant difference, and one even showed an advantage for the direct task.
- Lack of Consistent Evidence: Not a single study provided consistent evidence for ITAs across its conditions, despite all claiming them.
- Impact on Literature: The findings suggest that a large body of literature on unconscious priming may need to be reinterpreted, as the claims of superior unconscious processing are not robustly supported by the data when analyzed appropriately.
Limitations and Future Work
- Benefit-of-the-Doubt Approach: While strengthening their critique, the authors acknowledge that their reanalysis method is biased towards finding ITAs. They suggest that future research should use more balanced approaches or, ideally, trial-by-trial data.
- Need for Community Effort: The authors call for a broad reassessment of the unconscious priming literature by the scientific community, applying the appropriate analysis to substantiate or refute existing claims.
- Differentiating Conditions: They emphasize that the goal should be to identify the specific conditions (stimuli, tasks, setups) under which a reliable ITA might actually exist, rather than assuming its ubiquity.
- Beyond Behavioral Data: While their primary focus was on behavioral data, they note that their reanalysis method can be applied to neuroimaging data (EEG, fMRI) as well, which often have even higher noise levels.
Summary updated