Abstract: Despite the increasing popularity of Bayesian inference in empirical research, few practical guidelines provide detailed recommendations for how to apply Bayesian procedures and interpret the results. Here we offer specific guidelines for four different stages of Bayesian statistical reasoning in a research setting: planning the analysis, executing the analysis, interpreting the results, and reporting the results. The guidelines for each stage are illustrated with a running example. Although the guidelines are geared towards analyses performed with the open-source statistical software JASP, most guidelines extend to Bayesian inference in general.
Introduction
As empirical research increasingly adopts Bayesian methods, particularly in fields like psychology and biology, the need for standardized workflows has become paramount. Bayesian inference offers significant advantages over traditional frequentist null-hypothesis significance testing (NHST), such as the ability to quantify evidence in favor of a null hypothesis and the formal incorporation of prior knowledge. However, the relative novelty of these methods in applied settings often results in "Bayesian malpractice," where researchers may misinterpret credible intervals or fail to adequately justify their choice of prior distributions. This paper addresses this gap by providing a comprehensive, four-stage roadmap for researchers using the open-source software JASP to ensure their analyses are rigorous, transparent, and reproducible.
A Four-Stage Workflow for Bayesian Analysis
The proposed guidelines organize the Bayesian research process into four distinct stages: planning, execution, interpretation, and reporting.
1. Planning the Analysis
Before data collection or analysis begins, researchers must define their primary goal: is the objective to test a specific hypothesis (testing) or to determine the magnitude of an effect (estimation)? This stage involves choosing an appropriate statistical model and, crucially, specifying prior distributions. The choice of priors should be justified based on previous literature or theoretical expectations, rather than being left to software defaults without consideration.
2. Executing the Analysis
Execution involves data quality checks and the technical implementation of the model. Just as in frequentist statistics, researchers must assess data for normality and outliers. In JASP, this stage is facilitated by a graphical user interface that allows for real-time updates of results as different models or data transformations are applied. The guidelines emphasize the importance of visualizing data before running complex Bayesian models to avoid errors highlighted by classic examples like Anscombe’s Quartet.
3. Interpreting the Results
Interpretation requires a clear distinction between evidence and estimation. A Bayes factor ($BF$) provides a measure of relative evidence—how much more likely the data are under one hypothesis versus another. In contrast, a posterior distribution and its associated credible interval provide a range of plausible values for a parameter, assuming the effect exists. The guidelines caution against using credible intervals to perform hypothesis tests, as they serve different inferential purposes.
4. Reporting the Results
Transparency is the cornerstone of the reporting stage. Researchers are encouraged to include their .jasp files in supplementary materials to allow for full reproducibility. Reports should explicitly state the prior settings used and include robustness checks to show how sensitive the conclusions are to those settings.
Practical Application: The Stereogram Dataset
To illustrate these guidelines, the paper utilizes a classic dataset from Frisby and Clatworthy (1975). This study measured the "fuse time" (reaction time) required for participants to see a hidden figure in a stereogram, comparing a group that received visual information beforehand to a control group. This dataset is particularly useful for a tutorial because the raw data are non-normally distributed, necessitating data transformations or non-parametric Bayesian alternatives.
Applying the guidelines to this data, the analysis revealed "moderate evidence" ($BF_{+0} = 4.567$) that providing visual information reduces fuse time. The 95% credible interval for the effect size $\delta$ was found to be between 0.046 and 0.904, providing a clear estimate of the effect's magnitude alongside the evidence for its existence.
Key Insights and Reporting Standards
Testing vs. Estimation
A central theme of the guidelines is the independence of testing and estimation. While a Bayes factor might indicate strong evidence for an effect, the credible interval might still be wide, indicating low precision in the estimate. Conversely, a narrow credible interval does not automatically imply strong evidence against a null hypothesis.
Prior Robustness
Because priors can be perceived as subjective, the authors advocate for the use of Bayes factor robustness plots. These visualizations demonstrate how the $BF$ changes across a range of different prior widths. If the conclusion (e.g., "moderate evidence for the alternative") remains consistent across a wide range of priors, the result is considered "sturdy." If the conclusion shifts drastically with minor changes to the prior, it is "fragile" and should be interpreted with caution.
Standardized Notation
To avoid ambiguity, the guidelines recommend specific reporting standards:
- Subscripts: Always use subscripts to indicate the direction of the test (e.g., $BF_{10}$ for alternative vs. null, or $BF_{+0}$ for a one-sided positive effect).
- Evidence Categories: While evidence is continuous, using Jeffreys’ (1939) classification (e.g., $BF > 3$ as moderate, $BF > 10$ as strong) provides a helpful rule of thumb for communication.
Conclusion and Future Directions
While JASP serves as an excellent bridge for researchers transitioning to Bayesian methods, it is not without limitations. It currently focuses on a set of "default" priors and may not yet accommodate highly complex, custom-coded multidimensional models. Furthermore, the ease of use provided by a graphical interface carries the risk of "black box" analysis; researchers must remain vigilant in understanding the underlying logic of their models to avoid "garbage in, garbage out" scenarios.
As Bayesian methods continue to evolve, these guidelines will likely need to be updated to include more flexible tools like the BayesFactor R package or probabilistic programming languages such as Stan and PyMC3. Nevertheless, the current roadmap provides a vital foundation for improving the transparency and quality of Bayesian reporting in the social and natural sciences.
Summary updated