Similarities in fundamental frequency in infant speech segmentation models

Ellen Marklund, Francisco Lacerda, Iris-Corinna Schwarz, Ulla SundbergInterspeech 2012

TL;DR

This paper explores using fundamental frequency (F0) as a cue for segmenting speech in computational models of infant language acquisition. They found that F0-based segmentation is more effective for infant-directed speech (IDS) than for adult-directed speech (ADS), suggesting F0 plays a crucial role in how infants learn to identify word boundaries.

What Problem Does This Paper Address?

Learning to segment continuous speech into meaningful units like words is a fundamental challenge for infants acquiring language. While adults often speak in isolated words or clearly demarcated phrases, infants are typically exposed to a continuous stream of sound. Computational models aim to replicate this process, but they need to understand what cues infants actually use. Previous research has identified various cues like prosody, transitional probabilities, and phonotactics. This paper specifically investigates whether fundamental frequency (F0), a key prosodic feature that is often exaggerated in speech directed at infants (infant-directed speech or IDS), can serve as a primary cue for speech segmentation in models designed to mimic early language learning. The authors hypothesize that F0 might be particularly important for younger infants who are still developing their linguistic abilities.

Methods

The researchers developed a word-recognition segmentation procedure that uses an analytic approach, meaning it doesn't rely on pre-existing linguistic knowledge. They compared three methods for identifying similar speech segments:

  1. Fundamental Frequency (F0) based: Segments were considered similar if their F0 contours were alike.
  2. Transcription based: Segments were considered similar if their orthographic transcriptions matched.
  3. Randomly selected: Segments were chosen at random to serve as a baseline.

To quantify similarity, they calculated the "spectral distance" between segment pairs. This measure, expressed in Bark (a unit related to human auditory perception), represents the accumulated difference in the first four formants at equidistant points along the segments. A smaller spectral distance indicates greater acoustic similarity. They applied these segmentation bases to three types of speech: infant-directed speech for 3-month-olds (IDS03), infant-directed speech for 12-month-olds (IDS12), and adult-directed speech (ADS).

Datasets

The study used speech recordings from three Swedish female native speakers. The datasets included:

  • Infant-Directed Speech (IDS03): Speech directed to 3-month-old infants.
  • Infant-Directed Speech (IDS12): Speech directed to 12-month-old infants.
  • Adult-Directed Speech (ADS): Speech directed to adults.

The IDS was recorded in a sound-attenuated room with mothers interacting naturally with their infants. The ADS was elicited by an experimenter conversing with the mothers. The total durations of the speech material were: IDS03 (11 min 58 s), IDS12 (16 min 50 s), and ADS (10 min 29 s).

Key Insights and Ideas

  • F0 is a strong cue for infant speech segmentation: The study strongly suggests that fundamental frequency is a significant and viable cue for models aiming to simulate how infants segment speech, especially when dealing with infant-directed speech.
  • IDS characteristics matter: The exaggerated prosodic features of IDS, particularly F0 variations, seem to provide a distinct advantage for segmentation compared to the more neutral prosody of ADS.
  • Age-appropriateness of models: The effectiveness of F0 as a segmentation cue appears to be more pronounced for younger infants (IDS03) than for older infants (IDS12), hinting at developmental changes in how infants utilize acoustic information.
  • Limitations of transcription-based segmentation for models: While transcriptions are useful for human analysis, relying solely on them for infant models might overlook crucial acoustic information that infants naturally process.

Key Results

The study found significant differences in mean spectral distance across segmentation bases for both IDS conditions:

  • IDS03: A significant difference was found between segmentation bases (F(2,47)=9.679, p<0.001). Specifically, F0-based segmentation resulted in significantly smaller spectral distances compared to transcription-based segmentation (p<0.001).
  • IDS12: A significant difference was also observed (F(2,74)=3.838, p<0.026). F0-based segmentation showed a significantly smaller spectral distance than transcription-based segmentation (p < 0.007).
  • ADS: No significant difference was found between the segmentation bases (F(2,74)=0.257, p <0.774).

In essence, for infant-directed speech, segments identified using F0 similarity were acoustically more similar to each other than segments identified by matching transcriptions. This was not the case for adult-directed speech.

Limitations and Future Work

Limitations: * Memory Simulation: The segmentation process used a crude memory simulation by comparing utterances only within short sections, implying perfect recall within those sections and total memory loss between them. The authors suggest that incremental processing would offer a more nuanced memory simulation. * Unbalanced Speaker Distribution: Comparisons between speech style conditions were not made due to an unbalanced distribution of speakers across the different speech types.

Future Work: * The authors suggest that fundamental frequency is a promising avenue for modeling infant speech segmentation, particularly for IDS. * Further research could explore more sophisticated memory simulation techniques. * Investigating the interplay of F0 with other prosodic and segmental cues in IDS would be valuable.

Summary updated

BibTeX
@inproceedings{marklund2012similarities,
  author = {Ellen Marklund and Francisco Lacerda and Iris-Corinna Schwarz and Ulla Sundberg},
  title = {{Similarities in fundamental frequency in infant speech segmentation models}},
  booktitle = {Interspeech},
  year = {2012},
}
Figure