PredPsych: A toolbox for predictive machine learning-based approach in experimental psychology research

Atesh Koul, Cristina Becchio, Andrea Cavallo

Abstract

Recent years have seen an increased interest in machine learning-based predictive methods for analyzing quantitative behavioral data in experimental psychology. While these methods can achieve relatively greater sensitivity compared to conventional univariate techniques, they still lack an established and accessible implementation. The aim of current work was to build an open-source R toolbox - "PredPsych" - that could make these methods readily available to all psychologists. PredPsych is a user-friendly, R toolbox based on machine-learning predictive algorithms. In this paper, we present the framework of PredPsych via the analysis of a recently published multiple-subject motion capture dataset. In addition, we discuss examples of possible research questions that can be addressed with the machine-learning algorithms implemented in PredPsych and cannot be easily addressed with univariate statistical analysis. We anticipate that PredPsych will be of use to researchers with limited programming experience not only in the field of psychology, but also in that of clinical neuroscience, enabling computational assessment of putative bio-behavioral markers for both prognosis and diagnosis.

Introduction

Experimental psychology has traditionally focused on explaining human behavior through causal mechanisms, typically utilizing univariate statistics like ANOVA or t-tests. However, these methods often struggle with the high-dimensional, multivariate nature of behavioral data and are not optimized for predicting future observations. While machine learning (ML) offers a solution through its focus on minimizing prediction error and identifying holistic patterns, its adoption in psychology has been hindered by a steep learning curve and the lack of a unified framework.

PredPsych fills this gap by wrapping complex algorithms into accessible R functions. It allows researchers to move beyond traditional statistics to achieve higher sensitivity and predictive power in behavioral data analysis.

The PredPsych Framework

The toolbox integrates several standard and specialized ML techniques into a single workflow, designed to answer specific research questions common in experimental psychology.

The PredPsych toolbox architecture, illustrating how specific research questions are addressed by various machine learning algorithms.

Key features of the toolbox include:

  • Classification: Implementation of Linear Discriminant Analysis (LDA), Support Vector Machines (SVM), and Decision Trees to determine if experimental conditions can be reliably predicted from behavioral data.
  • Validation and Significance: Robust cross-validation schemes (e.g., k-fold, Leave-One-Subject-Out) and permutation testing to ensure results are statistically significant and not due to chance.
  • Feature Selection: Use of Fisher scores to identify which specific variables contribute most to the discrimination between conditions.
  • Unsupervised Learning: Model-based clustering and dimensionality reduction (PCA/MDS) to explore intrinsic data variability.

Case Study: Motion Capture Analysis

The authors demonstrate the toolbox using a multiple-subject motion capture dataset. Participants performed reach-to-grasp movements toward two differently sized objects: a small hazelnut and a large grapefruit. The resulting dataset consisted of 848 observations across 121 kinematic variables, such as wrist velocity, height, and grip aperture.

Key Results and Interpretability

Using the motion capture dataset, PredPsych showcased the advantages of a predictive approach over traditional group-mean statistics:

  • Multivariate Sensitivity: While individual markers might not show significant differences, the combination of markers allowed an SVM model to achieve 65% accuracy in predicting object size early in the movement.
  • Statistical Rigor: Permutation testing (1,000 simulations) confirmed that classification results were highly significant (p = 0.001).
  • Interpretability: By including Decision Trees, the toolbox makes the "black box" of ML transparent for psychological theory-building.
Decision tree visualization showing the hierarchical kinematic features used to predict object size early in the reach.

As shown in the decision tree, the model provides clear "if-then" logic. For example, if the index finger's y-coordinate exceeds 59.82 mm early in the reach, the movement is directed toward the large object with 92.3% accuracy.

Conclusion

PredPsych enables researchers with limited programming experience to apply advanced computational methods to behavioral data. By shifting the focus from "goodness of fit" to "predictive power," the toolbox facilitates the discovery of bio-behavioral markers essential for both prognosis and diagnosis in clinical and experimental settings.

Summary updated

Abstract

Recent years have seen an increased interest in machine learning-based predictive methods for analyzing quantitative behavioral data in experimental psychology. While these methods can achieve relatively greater sensitivity compared to conventional univariate techniques, they still lack an established and accessible implementation. The aim of current work was to build an open-source R toolbox - "PredPsych" - that could make these methods readily available to all psychologists. PredPsych is a user-friendly, R toolbox based on machine-learning predictive algorithms. In this paper, we present the framework of PredPsych via the analysis of a recently published multiple-subject motion capture dataset. In addition, we discuss examples of possible research questions that can be addressed with the machine-learning algorithms implemented in PredPsych and cannot be easily addressed with univariate statistical analysis. We anticipate that PredPsych will be of use to researchers with limited programming experience not only in the field of psychology, but also in that of clinical neuroscience, enabling computational assessment of putative bio-behavioral markers for both prognosis and diagnosis.
BibTeX
@inproceedings{koul2017predpsych,
  author = {Atesh Koul and Cristina Becchio and Andrea Cavallo},
  title = {{PredPsych: A toolbox for predictive machine learning-based approach in experimental psychology research}},
  booktitle = {Behavior},
  year = {2017},
  doi = {10.3758/s13428-017-0987-2},
}
Figure