Abstract
Psychological embeddings provide a powerful formalism for characterizing human-perceived similarity among members of a stimulus set. Obtaining high-quality embeddings can be costly due to algorithm design, software deployment, and participant compensation. This work aims to advance state-of-the-art embedding techniques and provide a comprehensive software package that makes obtaining high-quality psychological embeddings both easy and relatively efficient. Contributions are made on four fronts: flexible trial configurations, a likelihood model for multiple similarity kernels, an active selection algorithm for efficient data collection, and group-specific attention weight parameters. A series of experiments highlight the impact of these contributions on converging to high-quality embeddings. The tools are available as the Python package PsiZ.
Introduction
How do humans perceive the relationship between a sparrow and a finch versus a sparrow and a hawk? In cognitive science, these internal "mental maps" are represented as psychological embeddings—low-dimensional spaces where distance corresponds to perceived dissimilarity. While foundational, building these maps is traditionally expensive and technically fragmented.
Standard Multidimensional Scaling (MDS) faces several challenges: extreme data inefficiency requiring massive numbers of simple comparisons, rigid models that ignore group-specific differences (e.g., experts vs. novices), and a lack of integrated software solutions. Roads and Mozer (2019) introduce PsiZ, a Python-based framework designed to bridge the gap between psychological theory and machine learning efficiency.
Methodology and Technical Contributions
The PsiZ framework introduces a generative model that jointly infers feature representations (item coordinates) and similarity function parameters.
Flexible Trial Configurations
Traditional methods often rely on simple "triplet" trials (e.g., "Is A more like B or C?"). PsiZ supports more complex tasks, such as "8-choose-2" rankings, which extract significantly more information from each participant interaction.
Active Selection and Joint Kernel Learning
To maximize efficiency, the system employs a Bayesian active selection algorithm. By calculating Expected Information Gain, the model proposes trials that most effectively reduce uncertainty about item positions. Furthermore, the framework supports multiple similarity kernels (e.g., Exponential, Student's-t) and uses gradient descent via TensorFlow to optimize parameters.
Group-Specific Attention
Borrowing from the INDSCAL model, PsiZ can model multiple groups within a shared feature space while allowing for different "attention weights" per dimension. This allows the model to capture how experts might focus on different features than novices.
Experimental Results
The framework was validated using the Birds-16 dataset (208 images across 16 species) and synthetic simulations.
- Efficiency Gains: Complex "8-choose-2" trials were found to be approximately 3x more efficient than standard triplets in terms of total participant time.
- Active Learning Impact: Combining complex trial formats with active selection resulted in a 5x speedup compared to random selection to reach high embedding quality (Pearson correlation of 0.95).
- Information Sharing: Modeling experts and novices in a shared space reached target accuracy in 14.2 worker hours, compared to 29.6 hours when modeled independently, demonstrating the power of shared feature representations.
Conclusion and Future Work
PsiZ provides a robust, end-to-end solution for inferring psychological embeddings. While challenges remain in automated dimensionality selection and handling human noise, the framework significantly lowers the barrier to entry for high-quality psychological modeling. Future directions include extending the framework to non-visual modalities and integrating these embeddings into human-in-the-loop machine learning tasks.
The PsiZ package is available on GitHub at https://github.com/roads/psiz.
Summary updated