Unifying Amplitude and Phase Analysis: A Compositional Data Approach to Functional Multivariate Mixed-Effects Modeling of Mandarin Chinese

Pantelis Z. Hadjipantelis, John A. D. Aston, Hans‐Georg Müller, Jonathan Paul EvansarXiv 2013 · DOI · arXiv

Abstract

Mandarin Chinese is a tonal language where the fundamental frequency ($F_0$) of utterances carries critical linguistic information. However, speech samples vary significantly across individuals in both amplitude (pitch height) and phase (timing). This paper presents a joint model for amplitude, phase, and duration that integrates Functional Data Analysis (FDA), Compositional Data Analysis (CoDA), and Linear Mixed-Effects (LME) models. By decomposing pitch contours via functional principal component analysis and applying a compositional transformation to registration functions, the researchers formulate a multivariate mixed-effect model. Applied to the COSPRO-1 dataset, the model reveals that phonetic information is jointly carried by both amplitude and phase variation, and that individual speaker characteristics often outweigh linguistic context in pitch variance.

The Challenge of Pitch Variation

In tonal languages like Mandarin, pitch is the primary carrier of meaning. Analyzing pitch is difficult because no two speakers produce the same sound identically. One speaker may have a naturally higher voice (amplitude variation), while another might stretch or compress specific parts of a syllable (phase variation).

Figure 1: Examples of raw F0 tracks for different speakers and tone sequences, illustrating the inherent variation in pitch height and timing.

Traditionally, linguists "time-normalize" speech by stretching syllables to a standard length. While this simplifies comparison, it discards "phase" information—the subtle timing cues that may be linguistically significant. The challenge lies in statistically modeling these intertwined types of variation alongside total duration without losing the data's characteristic shape.

Methodology: A Three-Pillar Framework

The authors propose a "Unifying Amplitude and Phase Analysis" framework built on three sophisticated statistical pillars:

  1. Functional Data Analysis (FDA): Pitch contours are treated as continuous mathematical functions rather than discrete data points.
  2. Compositional Data Analysis (CoDA): To handle "warping functions" (timing variations), the authors treat them as cumulative distribution functions. Using a centered log-ratio (clr) transform, they convert constrained timing data into an unconstrained space suitable for linear modeling.
  3. Multivariate Linear Mixed-Effects (LME) Models: After using Functional Principal Component Analysis (FPCA) to reduce complex curves into representative scores, these are plugged into a joint regression model. This allows for the simultaneous analysis of linguistic and non-linguistic factors.
Figure 4: Summary of fixed and random effects included in the multivariate mixed-effects model, including tonal context, speaker sex, and duration.

Data and Key Insights

The study utilized the COSPRO-1 dataset, containing approximately 54,707 annotated syllabic $F_0$ contours from five native Taiwanese Mandarin speakers. The analysis yielded several key insights:

  • Phase is Not Noise: Phonetic information is encoded in both amplitude and phase. Timing is a structured part of the language's "typography," not just random jitter.
  • The "Catch-up" Effect: Phase components often show a "slow start" followed by an "accelerated ending," suggesting speakers subconsciously manage a "time budget" within each syllable.
  • Biology vs. Context: A striking finding was that variance attributed to the Speaker (physiology/identity) was significantly larger than variance from the Sentence (linguistic context). For the primary pitch component, the Speaker effect was more than double the Sentence effect.
  • Pitch-Duration Correlation: Higher-pitched utterances tend to last longer, likely due to the increased physiological effort required to manifest higher $F_0$ placements.

Conclusion and Future Work

The joint model successfully reconstructed original pitch tracks with high fidelity, even when ignoring components below the human "Just Noticeable Difference" threshold. While the study highlights the dominance of individual speaker characteristics, the authors note that the small speaker sample (5 individuals) is a limitation. Future research may extend this framework to model how amplitude and warping functions evolve over entire sentences rather than treating syllables as independent units.

Summary updated

Abstract

Mandarin Chinese is characterized by being a tonal language; the pitch (or $F_0$) of its utterances carries considerable linguistic information. However, speech samples from different individuals are subject to changes in amplitude and phase which must be accounted for in any analysis which attempts to provide a linguistically meaningful description of the language. A joint model for amplitude, phase and duration is presented which combines elements from Functional Data Analysis, Compositional Data Analysis and Linear Mixed Effects Models. By decomposing functions via a functional principal component analysis, and connecting registration functions to compositional data analysis, a joint multivariate mixed effect model can be formulated which gives insights into the relationship between the different modes of variation as well as their dependence on linguistic and non-linguistic covariates. The model is applied to the COSPRO-1 data set, a comprehensive database of spoken Taiwanese Mandarin, containing approximately 50 thousand phonetically diverse sample $F_0$ contours (syllables), and reveals that phonetic information is jointly carried by both amplitude and phase variation.
BibTeX
@misc{hadjipantelis2013unifying,
  author = {Pantelis Z. Hadjipantelis and John A. D. Aston and Hans‐Georg Müller and Jonathan Paul Evans},
  title = {{Unifying Amplitude and Phase Analysis: A Compositional Data Approach to Functional Multivariate Mixed-Effects Modeling of Mandarin Chinese}},
  year = {2013},
  eprint = {1308.0868},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1308.0868},
}
Figure