TL;DR
This paper proposes a novel method to accurately synchronize speech and Electroglottography (EGG) signals by leveraging the phase information of Zero-Frequency Filtered (ZFF) speech and EGG signals. This precise synchronization is crucial for various speech processing applications and understanding vocal fold dynamics, especially in medical contexts.
What Problem Does This Paper Address?
Simultaneously recording speech and EGG signals is valuable for research in areas like voice conversion, speech synthesis, and emotion recognition. However, a natural time delay, known as the glottis-to-microphone delay, exists between these two signals. This delay arises because the speech signal travels from the glottis, through the vocal tract, and then to the microphone, while the EGG signal captures glottal activity more directly. This delay isn't constant; it varies due to natural speaker movements and microphone positioning. Previous methods often assumed a fixed delay, leading to imprecise synchronization. This paper aims to address this by proposing a method for accurate, frame-by-frame estimation and compensation of this glottis-to-microphone delay.
Methods
The core of the proposed method lies in extracting and comparing the phase information from both the EGG signal and a specially processed speech signal.
- Zero-Frequency Filtering (ZFF) of Speech: The authors first process the speech signal by applying a Zero-Frequency Filter (ZFF). They argue that the ZFF-speech signal retains similar glottal activity information as the EGG signal, despite differences in their physical origins.
- Phase Extraction: For both the EGG signal and the ZFF-speech signal, the amplitude envelope and phase are extracted using the Hilbert transform. The Hilbert transform is a standard signal processing technique used to derive the analytic signal, from which amplitude and phase can be computed.
- Phase Comparison and Delay Estimation: The key insight is that the phase of the ZFF-speech signal is expected to lag behind the phase of the EGG signal by an amount corresponding to the glottis-to-microphone delay. This lag is estimated by computing the cross-correlation between the phase of the EGG signal ($\Phi_e(n)$) and the phase of the ZFF-speech signal ($\Phi_a(n)$). The delay is determined at the sample point where this cross-correlation is maximum. This allows for frame-by-frame estimation of the delay.
Datasets
The proposed method was evaluated using the CMU-Arctic database. This database contains simultaneous recordings of speech and EGG signals from one female (F1) and two male (M1, M2) speakers. The paper notes that this database was previously used in literature for similar evaluations, often assuming a fixed delay (e.g., 0.7 ms).
Key Insights and Ideas
- Phase as a Proxy for Glottal Activity: The paper's central idea is that the phase of the ZFF-speech signal, much like the EGG signal, carries robust information about glottal activity. This is a clever way to find common ground between two signals that capture different physical phenomena.
- Dynamic Delay Estimation is Crucial: The research highlights that assuming a fixed glottis-to-microphone delay is a significant limitation. The proposed frame-by-frame approach allows for dynamic adaptation to variations in speaker movement and microphone placement, leading to much more accurate synchronization.
- ZFF-Speech Captures Glottal Dynamics: The effectiveness of the ZFF technique in preserving glottal information from the speech signal is a key enabler for this method. It effectively filters out vocal tract characteristics that might otherwise obscure glottal timing.
Key Results
- The proposed method successfully estimates the glottis-to-microphone delay.
- In one experiment, the estimated delay of 2.125 ms (34 samples at 16 kHz) corresponded to a physical distance of approximately 70.76 cm, which was close to the experimental setup's estimated distance.
- The evaluation on the CMU-Arctic database showed that the mean glottis-to-microphone delay across speakers was around 0.713 ms, comparable to previous fixed-delay assumptions but now estimated dynamically.
- Histograms of delays between Glottal Closure Instants (GCIs) before and after synchronization demonstrated a significant reduction in average delay and standard deviation, indicating improved synchronization. For speaker M1, the average delay reduced from -1.1 ms to -0.15 ms, and the standard deviation decreased from 0.27 ms to 0.15 ms.
Limitations and Future Work
- Segment Length: The paper acknowledges that even after frame-level synchronization (using 100 ms segments), there might still be a slight residual delay between GCIs. This is attributed to the fact that the delay correction is applied uniformly across all glottal cycles within a segment, and individual glottal cycle delays can fluctuate.
- Future Work: The authors suggest that the proposed method can be extended to synchronize speech and EGG signals at a finer level, such as for each individual glottal cycle. This would offer even greater precision for detailed analysis of vocal fold dynamics.
Summary updated