A Rational Analysis of the Speech-to-Song Illusion

Raja Marjieh, Pol van Rijn, Ilia Sucholutsky, Harin Lee, Thomas L. Griffiths, Nori JacobyarXiv 2024 · DOI · arXiv

Abstract

The "speech-to-song" illusion—where repeated spoken phrases are perceived as music—has long been viewed as an auditory curiosity. This study reinterprets the effect as a rational statistical inference problem. We demonstrate that the illusion persists in purely written text (the "prose-to-lyrics" illusion), suggesting that the brain categorizes stimuli based on the statistical likelihood of repetition in song versus ordinary communication. Using Bayesian modeling, human behavioral experiments, and large language model (LLM) comparisons, we show that repetition serves as a strong cue for musicality because it is statistically more common in music than in prose.

Introduction

First identified by Diana Deutsch in 2003, the speech-to-song illusion occurs when a looped recording of a spoken phrase (e.g., "sometimes behave so strangely") begins to sound like it is being sung. While the effect is robust, existing theories—often focusing on low-level auditory processing or pitch-processing fatigue—fail to explain why the effect is selective. Some phrases transform into song almost instantly, while others remain perceived as speech regardless of repetition.

We propose that this illusion is a "rational" decision made by the brain's internal Bayesian processor. When encountering highly repetitive stimuli, the brain determines it is more likely that the stimulus originated from a generative model of music (where repetition is a core feature) than from a model of speech (where it is rare).

Conceptual overview of the Speech-to-Song and Prose-to-Lyrics illusions.

Methods

Rational Analysis Framework

We modeled human cognition as an optimal solution to a categorization problem using a Bayesian log-odds ratio: $$\log \frac{P(\text{song}|s)}{P(\text{speech}|s)}$$ This determines if a stimulus $s$ is more likely to be "song" or "speech" based on its internal structure.

Experimental Design

  1. Textual Analysis: To isolate the effect from auditory cues, we applied the model to text using a "bag-of-words" approach, comparing word repetition frequencies in song lyrics (2,994 samples from Shazam) versus spoken transcripts (948 samples from COCA).
  2. Human Behavioral Study: 120 participants rated text transcripts (repeated 0 to 4 times) on a scale from "definitely speech" to "definitely song."
  3. LLM Comparison: We replicated the behavioral experiment using GPT-4 to determine if a model trained on human text distributions exhibits similar categorization biases.

Results

The Prose-to-Lyrics Illusion

The study confirms that the illusion does not require sound. Simply reading a sentence repeated multiple times causes both humans and LLMs to perceive it as more likely to be a song lyric.

Human and LLM Alignment

In behavioral trials, the mean rating for sentences shifted from significantly "speech-like" (0.31) at zero repetitions to significantly "song-like" (0.58) after four repetitions. GPT-4’s ratings closely mirrored human judgments ($r = 0.64$), suggesting that the illusion is a byproduct of learning the statistical structures of language and music.

Comparison of mean song-to-speech ratings between human participants and GPT-4 across repetitions.

Phrase Sensitivity

The researchers identified a significant "sentence-repetition interaction." Phrases common in lyrics (e.g., "I love you") reached high song-likeness scores rapidly. In contrast, generic or technical phrases (e.g., "I drilled a hole through the top of my rack") remained perceived as speech for much longer, even with repetition.

Discussion

Our findings suggest that the speech-to-song illusion is a manifestation of a broader statistical inference mechanism. The "heavy tails" of repetition in musical corpora provide a rational basis for the brain to reclassify repetitive speech as song.

Future work will move beyond the bag-of-words model to incorporate rhythmic structure and sequence probability using more sophisticated generative models. Additionally, applying this rational analysis to audio—by training models on pitch ($f_0$) contours—may provide a unified explanation for both the auditory and textual versions of the illusion.

Summary updated

Abstract

The speech-to-song illusion is a robust psychological phenomenon whereby a spoken sentence sounds increasingly more musical as it is repeated. Despite decades of research, a complete formal account of this transformation is still lacking, and some of its nuanced characteristics, namely, that certain phrases appear to transform while others do not, is not well understood. Here we provide a formal account of this phenomenon, by recasting it as a statistical inference whereby a rational agent attempts to decide whether a sequence of utterances is more likely to have been produced in a song or speech. Using this approach and analyzing song and speech corpora, we further introduce a novel prose-to-lyrics illusion that is purely text-based. In this illusion, simply duplicating written sentences makes them appear more like song lyrics. We provide robust evidence for this new illusion in both human participants and large language models.
BibTeX
@misc{marjieh2024rational,
  author = {Raja Marjieh and Pol van Rijn and Ilia Sucholutsky and Harin Lee and Thomas L. Griffiths and Nori Jacoby},
  title = {{A Rational Analysis of the Speech-to-Song Illusion}},
  year = {2024},
  eprint = {2402.06992},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2402.06992},
}
Figure