Communicating compositional patterns

Eric Schulz, Francisco Quiroga, Samuel J. GershmanbioRxiv 2018 · DOI

TL;DR

This paper explores how humans communicate complex patterns, finding that patterns with a compositional structure (built from simpler parts) are significantly easier to describe and redraw than non-compositional ones. This suggests that our natural language for describing patterns leverages a compositional "grammar," similar to how we understand and build complex objects or ideas.

What Problem Does This Paper Address?

How do humans perceive, describe, and reconstruct complex patterns? While we intuitively understand that some patterns are easier to grasp and communicate than others, a formal understanding of why this is the case, especially concerning the role of structure, has been lacking. This paper tackles this by investigating whether a compositional structure in patterns makes them more communicable, and if a computational model of compositional pattern description can predict human performance.

Methods

The researchers designed a three-part communication game:

  1. Description: Participants described various patterns (some compositional, some not) to a hypothetical listener.
  2. Drawing: Another group of participants received these descriptions and attempted to redraw the original patterns.
  3. Rating: A third group rated the quality of the drawings compared to the original patterns.

They also employed a Gaussian Process (GP) framework to model pattern description. This involves defining "kernels" that represent different types of patterns. They compared a compositional kernel grammar (combining basic kernels like linear, radial basis, and periodic) against a non-compositional kernel (spectral mixture). This allowed them to computationally assess pattern complexity and compare it to human performance.

Datasets

The study used patterns generated from both compositional and non-compositional kernels. These patterns were sampled and then matched based on spectral entropy and wavelet distance to ensure a fair comparison between the two types. The paper does not introduce a new benchmark dataset but rather uses generated patterns for experimental purposes.

Key Insights and Ideas

  • Compositionality is Key to Communication: The core finding is that patterns built from simpler, repeatable components (compositional) are significantly easier for humans to communicate than those without such inherent structure.
  • Natural Language Mirrors Compositional Structure: The study suggests that the way we use language to describe patterns reflects this compositional nature, using more abstract and metaphorical terms for compositional patterns.
  • A Computational Model Predicts Human Performance: A compositional model of pattern description (using Gaussian Processes) accurately predicts human difficulty in describing and redrawing patterns, as well as how those drawings are rated.
  • Beyond Smoothness: While smoothness is a common inductive bias in machine learning, this paper shows that compositional structure is a more powerful predictor of human pattern communication than mere smoothness.

Key Results

  • More Accurate Drawings: Participants were more accurate at redrawing compositional patterns, as measured by both absolute error and wavelet distance.
  • Higher Quality Ratings: Human raters consistently gave higher quality scores to drawings of compositional patterns.
  • Lexical Diversity: Descriptions of compositional patterns exhibited higher lexical diversity, suggesting a richer vocabulary is used to capture their structure.
  • Model Fit: The compositional model's ability to "compress" patterns (measured by log-likelihood) was a strong predictor of how well humans could communicate those patterns, a relationship not found for the non-compositional model.

Limitations and Future Work

  • Natural Language Encoding: The paper lacks a computational account of how patterns are encoded into natural language. Future work could explore machine learning techniques to find invertible mappings between structural descriptions and natural language.
  • Decoding Descriptions: Similarly, a computational model for decoding natural language descriptions back into patterns for redrawing is missing.
  • Limited Function Set: The study used a relatively small set of functions. Future research should explore a broader range to ensure generalizability.
  • Distinguishing Patterns: Clearly distinguishing between compositional and non-compositional patterns can be challenging, especially with limited data. Future work could focus on refining these distinctions.

Summary updated

Abstract

Abstract How do people perceive and communicate structure? We investigate this question by letting participants play a communication game, where one player describes a pattern, and another player redraws it based on the description alone. We use this paradigm to compare two models of pattern description, one compositional (complex structures built out of simpler ones) and one non-compositional. We find that compositional patterns are communicated more effectively than non-compositional patterns, that a compositional model of pattern description predicts which patterns are harder to describe, and that this model can be used to evaluate participants’ drawings, producing human-like quality ratings. Our results suggest that natural language can tap into a compositionally structured pattern description language.
BibTeX
@misc{schulz2018communicating,
  author = {Eric Schulz and Francisco Quiroga and Samuel J. Gershman},
  title = {{Communicating compositional patterns}},
  year = {2018},
  doi = {10.1101/451161},
  howpublished = {bioRxiv},
}

Loading related content...

Figure