Does Cue Coordination Make a Voice Sound Old?

Listeners judge a stranger's age from a few seconds of speech, and decades of phonetics work have catalogued the cues they use: lower pitch, lower first formant (a resonance of the vocal tract), slower speaking rate, breathier phonation. Nearly every model of perceived age treats these cues as separate ingredients. It averages each one over a passage and adds them up in a linear regression, or it manipulates one cue at a time in a synthesizer. Aging, however, moves the cues together. A longitudinal study found that pitch and first formant fell by almost the same proportion over fifty years in one speaker, in a coordinated shift that seems to preserve vowel identity. This proposal asks a question no study in the literature has asked directly. Does the coordination between cues, meaning how pitch, formants, phonation quality and timing rise and fall together within an utterance, carry information about perceived age beyond what the cue averages and spreads already carry? And do listeners actually use that coordination?

The core claim is that the joint covariance of a small set of interpretable cues is a distinct, measurable correlate of perceived age, and that it can be tested two ways. Joint covariance here means the full matrix of how each cue co-varies with every other cue, including across short time lags. The first test is predictive: it measures how much this matrix adds beyond means and variances. The second is causal: resynthesized speech keeps every cue's own distribution and rhythm intact but scrambles or transplants their coordination, and listeners rate how old it sounds. The covariance-modeling literature supplies the tool. It uses a compact structured form, a Kronecker-separable "matrix-variate" covariance that factors feature-by-feature correlation from lag-by-lag correlation, because full covariance overfits small corpora. The perceived-age literature supplies the question and the listeners.

What already exists and where this differs

Prior work splits cleanly along the line this proposal crosses. On the perception side, Modeling perceived vocal age in american English regresses listener-rated age on 33 static summaries (means and ranges) for 150 speakers and reaches correlations up to 0.92 for women and 0.88 for men. It motivates the proposal and also constrains it: the ceiling is already high, so any gain from coordination must be shown under strict cross-validation, not asserted. Synthesis studies manipulate cues one at a time. Testing the relevance of speech rate, pitch and a glottal Chink for the perception of age in synthesized speech using formant synthesis found speech rate and glottal leakage significant and static pitch not significant. Controlling formant frequencies with neural text-to-speech for the manipulation of perceived speaker age showed that formant scaling alone made 82% of listeners hear a voice as older. The authors of that file entry flag the "entanglement" of cues as an open problem. None of these studies holds marginals fixed while changing coupling, and that manipulation is the new experimental lever here.

The motivating physiological evidence comes from Age-related changes in fundamental frequency and formants: a longitudinal study of four speakers. It reports a 22.1% pitch drop and a 21.9% first-formant drop for Queen Elizabeth II and interprets the joint movement as a diagonal shift in speaker space. Those are long-term shifts between decades. Whether the same coupling shows up within an utterance as a perceptible signature is unknown. Listener estimation of speaker age based on whispered speech shows that 69% correct age-group assignment survives with no pitch at all, which sets up a natural sub-question: is the coordination among the remaining resonance cues enough?

On the modeling side, Application of matrix variate Gaussian mixture model to statistical voice conversion and Voice Conversion Based on Matrix Variate Gaussian Mixture Model Using Multiple Frame Features supply the method. Unrestricted full-covariance mixtures overtrain beyond 32 components, and a Kronecker structure over features and context frames stays stable. Both papers address speaker-identity conversion between adult males and never age. The second also notes the main limitation of the structured model, which cannot represent interactions specific to one feature at one lag; the ablations below test that limitation explicitly. Modeling spectral variability for the classification of depressed speech found that second-order variance statistics carry clinical information, though only diagonal variances were used. Robust and accurate features for detecting and diagnosing autism spectrum disorders found that full covariance over voice-quality features helps clinical detection. Together these suggest that off-diagonal structure is informative in paralinguistics. Closest in task is Speaker Age Classification and Regression Using i-Vectors. It uses covariance normalization on holistic 400-dimensional speaker vectors from a diagonal front-end, and it trains on chronological rather than perceived age. Its covariance handles nuisance variability rather than describing interpretable cue coupling, and it cannot say which coordination matters.

Research Program

The per-frame representation is deliberately small and interpretable. It has eight tracks: log pitch, the first three formants in Bark, a vocal-tract-length proxy derived from the formants, harmonic-to-noise ratio, a spectral tilt measure standing in for glottal leakage, and a local syllable-rate envelope. Each voiced region is stacked over a context of three lags at a coarse frame rate of about 50 ms, giving an 8×3 matrix per time step. For each speaker the work estimates three nested descriptions.

The first, the marginal description, holds the mean and variance of each track, the information a diagonal model or static-summary regression sees. The second, the structured joint description, is a Kronecker covariance $\Sigma = \Sigma_F \otimes \Sigma_L$. Here $\Sigma_F$ is the 8×8 cross-cue correlation and $\Sigma_L$ is the 3×3 lag correlation, fitted by the alternating "flip-flop" maximum-likelihood updates used in matrix-variate models. That is 36 + 6 parameters, against 300 for an unrestricted 24-dimensional covariance. The third is the unrestricted full covariance, shrunk toward the structured one. Covariance matrices do not live in a flat space, so each is mapped to a vector by the matrix logarithm (the log-Euclidean tangent map) before regression. The correlation part is separated from the variance part so that the marginal and coordination contributions can be attributed cleanly.

Perceived age is regressed with ridge regression under nested speaker-disjoint cross-validation, separately by sex given the documented asymmetry. Gender is not the only demographic thread. A second target is the residual of perceived minus chronological age, which tests whether coordination explains why some voices sound older than their years, a divergence the literature flags but no covariance-based age model has targeted.

The causal half uses a vocoder that represents speech as pitch, spectral envelope and aperiodicity, the STRAIGHT-style decomposition used in An investigation of acoustic features for singing voice conversion based on perceptual age. Formant movement is controlled through a per-frame frequency-warp series on the envelope. Two manipulations hold every track's mean, variance and autocorrelation fixed. Decoupling applies phase-randomized surrogates jointly to the warp and phonation series relative to pitch. This preserves each series' own power spectrum and destroys their cross-correlation. Transplanting whitens a speaker's joint trajectory with their own $\Sigma_F$ and recolors it with the group-average $\Sigma_F$ of the opposite age group, then restores the original marginals. Listeners rate perceived age in years. A modest secondary hypothesis comes from Characterization of voice cue sensitivity and vocal emotion recognition across the adult lifespan, which found pitch sensitivity declining with listener age while vocal-tract-length sensitivity stayed stable. If so, older listeners should be less affected by manipulations of pitch-involving coupling than by manipulations of formant-phonation coupling.

Baselines

The naive reference predicts each held-out speaker's perceived age as the training-fold mean for their sex. Every model must beat it on mean absolute error and correlation.

The strongest existing method on perceived age is the static-summary linear regression of Modeling perceived vocal age in american English. It reports $r$ up to 0.92 (female) and 0.88 (male) on passages, and 0.73–0.92 across speech materials. Those figures come from that paper's own fitting procedure, so the regression will be re-implemented on its 33-feature design, on the same corpus and folds, with ridge strength tuned in the same nested loop. Beating it means a statistically reliable gain in held-out $r$ and error when correlation-structure features are added to its feature set. Reaching parity with it from the 8-track model is not enough. If the corpus itself cannot be obtained, the fallback is the AIST humming database for singing, and the claim narrows accordingly.

For chronological-age context, the i-vector system of Speaker Age Classification and Regression Using i-Vectors reports MAE 9.68 years for i-vectors with covariance normalization and 8.46 fused on aGender. The 8.46 figure is the bar for the chronological-age variant on aGender. No reported result was found on Lacuna for i-vectors trained on perceived age, so an i-vector regressor trained on perceived labels will be run in-house as the opaque high-capacity competitor.

The key ablations isolate the ingredient. Marginals only is the diagonal model. Marginals plus full unrestricted covariance tests whether structure, not just second-order information, matters. Marginals plus Kronecker covariance tests separability. A null control pairs each speaker's marginals with covariance computed from a different same-age-group, same-sex speaker. If the shuffled pairing performs as well as the true pairing, the "coordination" features are only coding group membership.

Why not just…?

Why not just add pairwise product features to the linear regression? Products of cue deviations are covariance entries, so this alternative is close to the proposal's predictive half, and the proposal does not pretend otherwise. On 150 speakers the unrestricted version has hundreds of interaction terms. The full-versus-Kronecker ablation measures whether the structured form's data efficiency matters, mirroring the overtraining collapse beyond 32 mixtures in Application of matrix variate Gaussian mixture model to statistical voice conversion. If plain interactions with strong ridge shrinkage match the Kronecker model, the result will be reported as such. The scientific claim then rests on the resynthesis experiment, which no feature-engineering baseline can replace.

Why not just use richer static summaries, means plus variances? Modeling spectral variability for the classification of depressed speech shows that variances alone can carry a lot, and this is a real threat: perhaps older voices are simply more variable on each cue independently. The marginals-only ablation and the decoupling manipulation settle it directly. Decoupling leaves every variance untouched, so any shift in perceived age it produces cannot be a variance effect.

Why not just use i-vectors or a neural speaker embedding? The fused i-vector system's 8.46 MAE shows that holistic embeddings predict chronological age well and may implicitly absorb cue coupling. They cannot isolate coordination as a cause or say which coupling matters, and their covariance normalization targets channel nuisance. The in-house i-vector run on perceived labels tells whether the interpretable model gives up accuracy. A small gap would be acceptable given the explanatory payoff. A large one would mean coordination is a minor part of what embeddings capture.

Why not just scale formants and pitch in a TTS system? Controlling formant frequencies with neural text-to-speech for the manipulation of perceived speaker age already shifts perceived age by moving marginals, and for controlling age that may be sufficient. The proposal's question is different: whether coordination contributes once the marginals are held fixed. Marginal scaling cannot answer that by construction.

Decisive Checks

The work will be judged on three outcomes. First, on speaker-disjoint folds, adding the correlation part of the Kronecker covariance to the static-summary regression should raise held-out correlation with perceived age, and the true pairing should beat the shuffled-pairing control. Second, coordination features should explain variance in the perceived-minus-chronological residual that marginals do not. Third, and most decisive, decoupled resynthesis should shift mean perceived age relative to unmodified vocoded speech, and transplanting old-group coupling into young voices should raise perceived age, with marginals verified unchanged by re-measurement. Vocoded controls separate the effect from vocoder artifacts.

A null result on the third check, with the first check positive, would mean coordination is a statistical marker of age but not a perceptual cue. That is still a useful finding, because it would tell synthesis work it can safely manipulate cues independently. The whispered corpus extends the third check to the pitch-free case, testing whether formant-phonation coupling alone supports the 69% listener accuracy.