Obtaining psychological embeddings through joint kernel and metric learning

Brett D. Roads, Michael C. MozerBehavior 2019 · DOI

Abstract

Psychological embeddings provide a powerful formalism for characterizing human-perceived similarity among members of a stimulus set. Obtaining high-quality embeddings can be costly due to algorithm design, software deployment, and participant compensation. This work aims to advance state-of-the-art embedding techniques and provide a comprehensive software package that makes obtaining high-quality psychological embeddings both easy and relatively efficient. Contributions are made on four fronts: flexible trial configurations, a likelihood model for multiple similarity kernels, an active selection algorithm for efficient data collection, and group-specific attention weight parameters. A series of experiments highlight the impact of these contributions on converging to high-quality embeddings. The tools are available as the Python package PsiZ.

Introduction

How do humans perceive the relationship between a sparrow and a finch versus a sparrow and a hawk? In cognitive science, these internal "mental maps" are represented as psychological embeddings—low-dimensional spaces where distance corresponds to perceived dissimilarity. While foundational, building these maps is traditionally expensive and technically fragmented.

Standard Multidimensional Scaling (MDS) faces several challenges: extreme data inefficiency requiring massive numbers of simple comparisons, rigid models that ignore group-specific differences (e.g., experts vs. novices), and a lack of integrated software solutions. Roads and Mozer (2019) introduce PsiZ, a Python-based framework designed to bridge the gap between psychological theory and machine learning efficiency.

Methodology and Technical Contributions

The PsiZ framework introduces a generative model that jointly infers feature representations (item coordinates) and similarity function parameters.

Flexible Trial Configurations

Traditional methods often rely on simple "triplet" trials (e.g., "Is A more like B or C?"). PsiZ supports more complex tasks, such as "8-choose-2" rankings, which extract significantly more information from each participant interaction.

A standard triplet comparison trial where a user selects the most similar image to a reference.

Active Selection and Joint Kernel Learning

To maximize efficiency, the system employs a Bayesian active selection algorithm. By calculating Expected Information Gain, the model proposes trials that most effectively reduce uncertainty about item positions. Furthermore, the framework supports multiple similarity kernels (e.g., Exponential, Student's-t) and uses gradient descent via TensorFlow to optimize parameters.

A complex trial configuration (e.g., 8-choose-2) that provides significantly more information per trial than simple triplets.

Group-Specific Attention

Borrowing from the INDSCAL model, PsiZ can model multiple groups within a shared feature space while allowing for different "attention weights" per dimension. This allows the model to capture how experts might focus on different features than novices.

Experimental Results

The framework was validated using the Birds-16 dataset (208 images across 16 species) and synthetic simulations.

  • Efficiency Gains: Complex "8-choose-2" trials were found to be approximately 3x more efficient than standard triplets in terms of total participant time.
  • Active Learning Impact: Combining complex trial formats with active selection resulted in a 5x speedup compared to random selection to reach high embedding quality (Pearson correlation of 0.95).
  • Information Sharing: Modeling experts and novices in a shared space reached target accuracy in 14.2 worker hours, compared to 29.6 hours when modeled independently, demonstrating the power of shared feature representations.

Conclusion and Future Work

PsiZ provides a robust, end-to-end solution for inferring psychological embeddings. While challenges remain in automated dimensionality selection and handling human noise, the framework significantly lowers the barrier to entry for high-quality psychological modeling. Future directions include extending the framework to non-visual modalities and integrating these embeddings into human-in-the-loop machine learning tasks.

The PsiZ package is available on GitHub at https://github.com/roads/psiz.

Summary updated

Abstract

Psychological embeddings provide a powerful formalism for characterizing human-perceived similarity among members of a stimulus set. Obtaining high-quality embeddings can be costly due to algorithm design, software deployment, and participant compensation. This work aims to advance state-of-the-art embedding techniques and provide a comprehensive software package that makes obtaining high-quality psychological embeddings both easy and relatively efficient. Contributions are made on four fronts. First, the embedding procedure allows multiple trial configurations (e.g., triplets) to be used for collecting similarity judgments from participants. For example, trials can be configured to collect triplet comparisons or to sort items into groups. Second, a likelihood model is provided for three classes of similarity kernels allowing users to easily infer the parameters of their preferred model using gradient descent. Third, an active selection algorithm is provided that makes data collection more efficient by proposing comparisons that provide the strongest constraints on the embedding. Fourth, the likelihood model allows the specification of group-specific attention weight parameters. A series of experiments are included to highlight each of these contributions and their impact on converging to a high-quality embedding. Collectively, these incremental improvements provide a powerful and complete set of tools for inferring psychological embeddings. The relevant tools are available as the Python package PsiZ, which can be cloned from GitHub ( https://github.com/roads/psiz ).
BibTeX
@inproceedings{roads2019obtaining,
  author = {Brett D. Roads and Michael C. Mozer},
  title = {{Obtaining psychological embeddings through joint kernel and metric learning}},
  booktitle = {Behavior},
  year = {2019},
  doi = {10.3758/s13428-019-01285-3},
}
Figure