2D Immersed Boundary Method in Vocal Tract Acoustics: An Eulerian–Lagrangian Model for Simulation of Diphthongs

Rongshuai Wu, Debasish Ray Mohapatra, Sidney FelsInterspeech 2025

TL;DR

This paper introduces a novel 2D Immersed Boundary Method (IBM) for simulating the complex acoustics of dynamic vocal tracts, specifically for diphthongs. By decoupling the vocal tract's Lagrangian boundary from a fixed Eulerian grid, it overcomes the computational expense of remeshing and the geometric inaccuracies of staircased approximations, achieving high-fidelity spectral features comparable to more complex models.

What Problem Does This Paper Address?

Synthesizing realistic speech, especially diphthongs (sounds that transition between two vowels), requires accurately modeling the dynamic changes in the vocal tract's shape. Existing methods face significant challenges:

  • Finite Element (FE) models: While accurate, they are computationally very expensive, often requiring complete remeshing of the simulation domain every time the vocal tract shape changes. This makes real-time or complex dynamic simulations prohibitive.
  • Digital Waveguide Mesh (DWM) models: These models are faster but suffer from geometric inaccuracies. They approximate curved vocal tract boundaries with staircased steps on a grid, leading to discretization errors that can affect the acoustic output, especially at higher frequencies.

The core problem is finding a method that can accurately simulate dynamic vocal tract shapes for diphthongs without incurring the prohibitive computational cost of FE methods or the geometric inaccuracies of DWM models.

Methods

The authors propose a 2D Immersed Boundary Method (IBM) for vocal tract acoustics, building upon previous work. Here's a breakdown of their approach:

  • Immersed Boundary Method (IBM): This is the core innovation. Instead of meshing the fluid domain to conform to the vocal tract's shape, IBM treats the vocal tract boundary as an "immersed body" within a fixed computational grid (Eulerian domain). The boundary's motion is tracked using Lagrangian points.
  • Eulerian-Lagrangian Framework: The acoustic wave equations (governing pressure and velocity) are solved on the fixed Eulerian grid using a 2D Finite-Difference Time-Domain (FDTD) scheme. The vocal tract's dynamic shape is represented by Lagrangian points.
  • Forcing Terms: The interaction between the fluid (air) and the immersed boundary (vocal tract walls) is modeled using discrete forcing terms. These terms are applied to the wave equations on the Eulerian grid to enforce the boundary conditions imposed by the moving vocal tract.
  • Free Boundary Motion: A key advantage of IBM is that it allows the Lagrangian boundary to move freely within the fixed Eulerian grid without requiring remeshing. This significantly simplifies the simulation of dynamic geometries.
  • Smooth Boundary Approximation: The method avoids the staircasing artifacts of DWM by representing the vocal tract contour with Lagrangian points and using a smooth interpolation for the Dirac delta function, which distributes the forcing terms onto the grid.
  • Dynamic Geometry Interpolation: For diphthongs, the vocal tract shape transitions between two target vowels. This transition is achieved by interpolating the area functions of the initial and target vowels using a nonlinear half-wave sinusoidal trajectory. The tract length is also dynamically adjusted using the same interpolation method.

Datasets

  • Story's area functions for vowels [27]: This is a widely recognized dataset used to define the initial and target vocal tract shapes (specifically, sagittal widths) for the 2D model.
  • Electrolaryngograph (Lx) signals [7]: These signals, which measure vocal fold conductivity, were used as the source signal to approximate glottal flow during speech synthesis.
  • rtMRI data (mentioned as a source for 3D geometries): While not directly used for training their 2D model, the paper references rtMRI data [11, 12] as a source for realistic 3D vocal tract geometries, which their model aims to emulate in terms of acoustic output.
  • Recorded audio samples [15]: Used for comparative analysis of the synthesized diphthongs.

Key Insights and Ideas

  • IBM as a bridge: The Immersed Boundary Method offers a compelling way to bridge the gap between computationally expensive, high-fidelity FE models and faster but less accurate DWM models for dynamic vocal tract simulation.
  • Eliminating remeshing and staircasing: The core benefit of IBM is its ability to handle dynamic boundaries without the need for remeshing (like FE) or the geometric distortions of staircasing (like DWM). This is crucial for accurate diphthong synthesis.
  • Focus on low-frequency formants: The study highlights that while high-frequency spectral features might differ due to the simplified 2D geometry (lack of side branches), the low-frequency formants (F1 and F2), which are critical for vowel and diphthong intelligibility, are well-captured.
  • Effective interpolation for diphthongs: The use of a nonlinear half-wave sinusoidal trajectory for interpolating area functions proved effective in creating smooth transitions for diphthong synthesis.

Key Results

  • Spectrogram Comparison: The spectrograms generated by the 2D IBM model closely matched those of a high-fidelity 3D DWM model and recorded audio, particularly for the [au] and [er] diphthongs, at low frequencies.
  • Formant Trajectory Correlation: The F1 and F2 formant transition curves extracted from the synthesized diphthongs showed a strong statistical resemblance to those from the 3D DWM model and recorded audio.
  • Correlation Coefficient: The Pearson correlation coefficient for the F1/F2 transitions between the 2D IBM model and the 3D DWM/recorded audio exceeded 0.8, indicating high similarity in these crucial spectral features.
  • Computational Efficiency (Implied): While not explicitly quantified, the IBM approach inherently offers better computational efficiency than FE methods requiring remeshing, and it avoids the geometric approximation errors of DWM.

Limitations and Future Work

  • Simplified 2D Geometry: The model uses a 2D representation of the vocal tract, omitting features like side branches (e.g., nasal cavity) and the complexity of the lips. This simplification leads to discrepancies in high-frequency spectral features (above 3-4 kHz) compared to 3D models and real speech.
  • Future Work:
    • Vowel-Consonant-Vowel (VCV) utterances: Extending the model to simulate more complex VCV utterances.
  • Boundary Interpolation Approaches: Investigating the impact of different interpolation methods for tract geometries on the acoustic output.
  • 3D Extension: While this paper focuses on 2D, a natural extension would be to explore a 3D IBM approach for even higher fidelity.

Summary updated

BibTeX
@inproceedings{wu2025d,
  author = {Rongshuai Wu and Debasish Ray Mohapatra and Sidney Fels},
  title = {{2D Immersed Boundary Method in Vocal Tract Acoustics: An Eulerian–Lagrangian Model for Simulation of Diphthongs}},
  booktitle = {Interspeech},
  year = {2025},
}
Figure