TL;DR
This paper introduces Fine-Tracker, a computational model that demonstrates how durational cues in speech can be used to improve spoken-word recognition, mirroring human capabilities. By successfully disambiguating acoustically similar words and segments based on duration, Fine-Tracker provides strong support for the theory that timing information is crucial for how we understand speech.
What Problem Does This Paper Address?
Understanding spoken language is a complex process, especially when speech is ambiguous. For instance, the word "ham" can be embedded within "hamster," and distinguishing between them relies on subtle cues. Previous research has shown that humans use the duration of sounds and words to help resolve these ambiguities. However, computational models of spoken-word recognition have not always effectively incorporated this durational information. This paper aims to bridge that gap by investigating whether a computational model, like humans, can benefit from durational cues to improve its ability to recognize words in continuous speech.
Methods
The core of this work is the Fine-Tracker model, which operates on two levels:
- Prelexical Level: This level processes the raw acoustic signal using a set of artificial neural networks (Multi-Layer Perceptrons or MLPs). Each MLP is trained to extract specific "articulatory features" (AFs) such as voicing, place of articulation, and importantly, duration. These AFs are represented as continuous values between 0 and 1, providing a "soft" decision about the presence of a feature at any given time.
- Lexical Level: This level takes the sequence of AF feature vectors from the prelexical level and compares them against a lexicon of words. The lexicon itself is represented in terms of AF feature vectors. A key innovation here is the ability to configure the number of feature vectors per phoneme in the lexicon. This allows the model to explicitly encode durational differences between words or parts of words. For example, a longer word or segment can be represented by more feature vectors, effectively stretching its representation in time.
The model uses a "word search" module that compares prelexical features with lexical representations to find the most likely word sequence. This search is time-synchronous and breadth-first, allowing for flexible mapping of acoustic input to lexical representations.
Datasets
The paper utilizes acoustic stimuli from two existing behavioral studies:
- Simulation 1 (Lexical Embedding): Stimuli from Salverda et al. (2003) are used. These consist of manipulated Dutch sentences containing target words where the first syllable is either cross-spliced from a monosyllabic word (MONO condition) or from another recording of that same target word (CARRIER condition).
- Simulation 2 (Segment Durations): Stimuli from Shatzman & McQueen (2006) are used. These are Dutch ambiguous sentences where a word boundary is ambiguous, for example, between "eens pot" (once jar) and "een spot" (a spotlight). The ambiguity is manipulated by splicing the [s] sound differently, creating variations in its duration.
The lexicon used in Fine-Tracker contains 27,740 entries. For simulations, priors were applied to a subset of words (61 for Simulation 1, 42 for Simulation 2) to limit the search space.
Key Insights and Ideas
- Duration as a Disambiguation Cue: The most significant insight is that Fine-Tracker, by explicitly modeling durational information in its lexicon, can successfully disambiguate acoustically similar speech segments and words, mirroring human behavior.
- Lexicon Configuration is Key: The ability to adjust the number of feature vectors per phoneme in the lexicon is crucial. This allows the model to represent temporal differences, making it sensitive to duration.
- Support for Durational Theory: The model's success provides computational evidence supporting the theory that durational cues play a fundamental role in spoken-word recognition, not just as a secondary cue but as a primary mechanism for resolving ambiguity.
- Prelexical to Lexical Integration: The paper highlights the effective integration of prelexical acoustic information with lexical knowledge, demonstrating a plausible pathway for how speech is processed.
Key Results
- Simulation 1 (Lexical Embedding):
- In the "no duration" condition, the embedded word had the highest activation in 10 out of 28 stimuli.
- When durational information was incorporated into the lexicon, this improved to 18 out of 28 stimuli.
- A significant correlation was found between the per-stimulus difference in syllable duration and the model's performance, indicating that larger durational differences led to better disambiguation.
- Simulation 2 (Segment Durations):
- For 14 out of 20 stimuli, the target word had higher activation in the IDENT condition (shorter [s] duration) compared to the CROSS condition (longer [s] duration).
- A significant correlation was found between the per-stimulus difference in [s] duration and the model's performance, showing that Fine-Tracker could use segment duration to place word boundaries.
Limitations and Future Work
- Manual Lexicon Configuration: Currently, the number of feature vectors per phoneme in the lexicon is set manually. Future work could involve automating this process, perhaps by learning these durational representations directly from data.
- Other Acoustic Cues: The authors acknowledge that humans likely use other acoustic cues (e.g., formant frequency, assimilation) in addition to duration. Future research should explore incorporating these cues into Fine-Tracker to further enhance its modeling power.
- Statistical Significance: While improvements were observed, the statistical significance of the improvement in Simulation 1 was borderline (p=0.071). Further investigation with more data or refined statistical methods might be warranted.
Summary updated