Computational models of music perception and cognition I: The perceptual and cognitive processing chain

H.‐g. Purwins, Perfecto Herrera, Maarten Grachten, Amaury Hazan, Ricard Marxer, Xavier SerraPhysics 2008 · DOI

TL;DR

This paper reviews computational models of music perception and cognition, bridging neuroscience, psychology, and artificial intelligence. It outlines a hierarchical processing chain from auditory input to complex cognitive operations, emphasizing the need for integrated models to understand how humans make sense of music.

What Problem Does This Paper Address?

Understanding how humans perceive and process music is a complex challenge that spans multiple scientific disciplines. While traditional music theory offers insights, it often falls short in explaining the underlying cognitive and neural mechanisms. This paper aims to bridge this gap by reviewing existing computational models that attempt to explain music perception and cognition, drawing from neuroscience, psychology, cognitive science, artificial intelligence, and musicology. The authors seek to establish a framework for building comprehensive models that can account for the entire perceptual and cognitive processing chain involved in listening to music, from the initial transduction of sound to higher-level cognitive functions like memory and expectation.

Methods

The paper adopts a comprehensive review approach, synthesizing findings from various fields. It discusses:

  • Neuroscience: Examining brain imaging techniques (EEG, fMRI, PET) and invasive animal studies to understand the neural correlates of auditory processing and music cognition.
  • Psychoacoustics and Music Psychology: Reviewing experimental methods used to study perceptual thresholds, just noticeable differences (JNDs), and the psychological aspects of music perception, such as grouping and expectancy.
  • Computational Modeling: Discussing various modeling approaches, including:
  • Auditory Periphery Models: Simulating the initial stages of sound processing in the ear.
  • Neural Processing Principles: Exploring concepts like Hebbian learning and tonotopy.
  • Feature Extraction and Grouping Models: Describing how basic acoustic features are detected and grouped into meaningful entities (e.g., tones, melodies). This includes models based on spectral analysis, temporal autocorrelation, and spectrotemporal approaches, as well as computational auditory scene analysis (CASA) and oscillator models.
  • Memory and Attention Models: Reviewing computational frameworks for short-term and long-term memory, and attentional mechanisms.
  • High-Level Cognition Models: Discussing approaches to category formation (e.g., Self-Organizing Maps, Adaptive Resonance Theory) and integrated cognition modeling environments like ACT-R.
  • Expectancy Models: Exploring how information theory and statistical models can capture musical expectations.

Datasets

Not explicitly discussed as the paper is a review of existing models and methodologies rather than a presentation of new experimental results or a benchmark dataset.

Key Insights and Ideas

  • Hierarchical Processing Chain: Music perception and cognition can be understood as a hierarchical processing chain, starting from low-level auditory transduction and progressing to high-level cognitive functions.
  • Interdisciplinary Approach is Crucial: A complete understanding of music cognition requires integrating knowledge from neuroscience, psychology, AI, and musicology. No single discipline holds all the answers.
  • Modular Architecture of Music Processing: Evidence from brain imaging suggests a modular organization of music processing, with different brain regions specializing in various aspects of auditory and musical analysis.
  • The "Symbolic Grounding Problem" in Music: A significant challenge is connecting raw audio signals to abstract, internal representations. The role of music scores as a candidate for such representations is questioned.
  • Grouping Principles are Key: Gestalt principles (proximity, similarity, good continuation, common fate) are fundamental to how we segment and organize auditory scenes, and are crucial for understanding music.
  • Expectancy Plays a Major Role: Musical expectancy, driven by learned patterns and statistical regularities, is deeply intertwined with information-theoretic concepts like entropy and surprise, influencing our perception and emotional response.

Key Results

This paper is a review and does not present new experimental results. However, it synthesizes findings that support:

  • The localization of various musical abilities (pitch, harmony, rhythm) in specific brain regions.
  • The effectiveness of computational models in explaining phenomena like pitch detection (including missing fundamentals), auditory stream segregation, and grouping.
  • The utility of frameworks like ACT-R for modeling integrated cognitive processes in music.
  • The application of information-theoretic measures to analyze musical structure and predict listener expectations.

Limitations and Future Work

The authors acknowledge several limitations and suggest future directions:

  • Bridging the Gap: A major obstacle is the "symbolic grounding problem" – connecting raw audio signals to abstract internal representations.
  • Cultural Influence: While some categorical distinctions might be universal, many musical categories are culturally dependent, posing challenges for universal models.
  • Model Validation: Many models are validated on simplified stimuli or specific aspects of music. Further validation with complex, real-world musical material is needed.
  • Integration of High-Level and Low-Level Processes: The feedback loops between high-level cognitive processes (like attention) and low-level auditory processing need further exploration.
  • Role of Embodiment: The influence of motion-related cognitive processes on music listening and performance is highlighted as an area of growing interest.
  • Domain-Specific Models: The paper sets the stage for Part II, which will discuss domain-specific models for melody, rhythm, and harmony, aiming to further clarify questions about music perception and cognition.

Summary updated

BibTeX
@inproceedings{purwins2008computational,
  author = {H.‐G. Purwins and Perfecto Herrera and Maarten Grachten and Amaury Hazan and Ricard Marxer and Xavier Serra},
  title = {{Computational models of music perception and cognition I: The perceptual and cognitive processing chain}},
  booktitle = {Physics},
  year = {2008},
  doi = {10.1016/j.plrev.2008.03.004},
}
Figure