Abstract
Mandarin Chinese is a tonal language where the fundamental frequency ($F_0$) of utterances carries critical linguistic information. However, speech samples vary significantly across individuals in both amplitude (pitch height) and phase (timing). This paper presents a joint model for amplitude, phase, and duration that integrates Functional Data Analysis (FDA), Compositional Data Analysis (CoDA), and Linear Mixed-Effects (LME) models. By decomposing pitch contours via functional principal component analysis and applying a compositional transformation to registration functions, the researchers formulate a multivariate mixed-effect model. Applied to the COSPRO-1 dataset, the model reveals that phonetic information is jointly carried by both amplitude and phase variation, and that individual speaker characteristics often outweigh linguistic context in pitch variance.
The Challenge of Pitch Variation
In tonal languages like Mandarin, pitch is the primary carrier of meaning. Analyzing pitch is difficult because no two speakers produce the same sound identically. One speaker may have a naturally higher voice (amplitude variation), while another might stretch or compress specific parts of a syllable (phase variation).
Traditionally, linguists "time-normalize" speech by stretching syllables to a standard length. While this simplifies comparison, it discards "phase" information—the subtle timing cues that may be linguistically significant. The challenge lies in statistically modeling these intertwined types of variation alongside total duration without losing the data's characteristic shape.
Methodology: A Three-Pillar Framework
The authors propose a "Unifying Amplitude and Phase Analysis" framework built on three sophisticated statistical pillars:
- Functional Data Analysis (FDA): Pitch contours are treated as continuous mathematical functions rather than discrete data points.
- Compositional Data Analysis (CoDA): To handle "warping functions" (timing variations), the authors treat them as cumulative distribution functions. Using a centered log-ratio (clr) transform, they convert constrained timing data into an unconstrained space suitable for linear modeling.
- Multivariate Linear Mixed-Effects (LME) Models: After using Functional Principal Component Analysis (FPCA) to reduce complex curves into representative scores, these are plugged into a joint regression model. This allows for the simultaneous analysis of linguistic and non-linguistic factors.
Data and Key Insights
The study utilized the COSPRO-1 dataset, containing approximately 54,707 annotated syllabic $F_0$ contours from five native Taiwanese Mandarin speakers. The analysis yielded several key insights:
- Phase is Not Noise: Phonetic information is encoded in both amplitude and phase. Timing is a structured part of the language's "typography," not just random jitter.
- The "Catch-up" Effect: Phase components often show a "slow start" followed by an "accelerated ending," suggesting speakers subconsciously manage a "time budget" within each syllable.
- Biology vs. Context: A striking finding was that variance attributed to the Speaker (physiology/identity) was significantly larger than variance from the Sentence (linguistic context). For the primary pitch component, the Speaker effect was more than double the Sentence effect.
- Pitch-Duration Correlation: Higher-pitched utterances tend to last longer, likely due to the increased physiological effort required to manifest higher $F_0$ placements.
Conclusion and Future Work
The joint model successfully reconstructed original pitch tracks with high fidelity, even when ignoring components below the human "Just Noticeable Difference" threshold. While the study highlights the dominance of individual speaker characteristics, the authors note that the small speaker sample (5 individuals) is a limitation. Future research may extend this framework to model how amplitude and warping functions evolve over entire sentences rather than treating syllables as independent units.
Summary updated