Abstract
Recent studies have shown various kinds of recurrent neural networks (RNNs) are becoming powerful sequence models in speech-related applications. Our previous work in detecting questions of Mandarin speech presents that gated recurrent unit (GRU) based RNN can achieve significantly better results. In this paper, we try to open the black box to find the correlations between inner architecture of GRU and phonetic features of question sentences. We find that both update gate and reset gate in GRU blocks react when people begin to pronounce a word. According to the reactions, experiments are conducted to show the behavior of GRU based question detection approach on three important factors, including keywords or special structure of questions, final particles and interrogative intonation. We also observe that update gate and reset gate don’t collaborate well on our dataset. Based on the asynchronous acts of update gate and reset gate in GRU, we adapt the structure of GRU block to our dataset and get further performance improvement in question detection task.
Introduction
In Mandarin Chinese, detecting questions is a complex task because the word order of a question is often identical to that of a statement. Detection primarily relies on lexical factors, such as interrogative pronouns or final particles (e.g., "ma"), and acoustic factors, specifically the "rising boundary tone" at the end of a sentence. While Recurrent Neural Networks (RNNs) like LSTMs and GRUs have proven highly effective at detecting these questions from raw acoustic features, they are often treated as "black boxes." This study aims to interpret the internal mechanisms of GRUs and optimize their architecture for speech-based question detection.
Interpretability Analysis
To understand how GRUs process speech, the researchers identified the most influential hidden nodes by analyzing the weights assigned to them in the final classification layer. By plotting the activations of the update and reset gates against the speech signal, several key insights emerged:
- Gate Reaction to Word Boundaries: The update gate tends to "open" (output near 0) at the onset of a new word, allowing the node to ingest new information, and "closes" (output near 1) during the latter half of the word to maintain memory.
- End-of-Sentence Bias: The model places the highest importance on the end of the utterance. This aligns with linguistic theory, as Mandarin question markers (particles and rising intonation) are typically found at the very end of a sentence.
- Redundancy: Observations indicated that the reset gate often duplicates the functionality of the update gate or the recurrent weight matrix, suggesting the standard GRU might be over-engineered for this specific task.
Architectural Optimization
Based on the observed redundancy and asynchronous behavior of the gates, the authors proposed two simplified models to improve efficiency and accuracy:
- S-GRU (Simplified GRU): This model sets the reset gate to a constant value of 1.
- SC-GRU (Single Connection GRU): This variant removes the recurrent connection from the candidate hidden state update, streamlining the information flow.
Experimental Results
The study utilized a simulated Call Center Recording dataset of Mandarin, featuring 5,700 utterances. Low-level descriptors (LLDs) including pitch and energy were extracted as features. The proposed SC-GRU was compared against standard RNNs, LSTMs, and GRUs with a controlled parameter count (~37k).
| Model | Number of Hidden Units | Time per Epoch (s) | F1 Measure (Mean) |
|---|---|---|---|
| S-RNN | 138 | 12 | 0.767 |
| LSTM | 52 | 29 | 0.796 |
| GRU | 64 | 22 | 0.801 |
| SC-GRU | 102 | 15 | 0.805 |
The SC-GRU achieved the highest F1 score while being significantly faster to train than the standard GRU and nearly twice as fast as the LSTM.
Conclusion
By "opening the black box," this research demonstrates that GRU gates effectively learn to track word boundaries and focus on sentence-final prosodic cues in Mandarin. The proposed SC-GRU architecture offers a more efficient alternative for question detection. However, the model still struggles with lexical keywords in the absence of prosodic cues, suggesting that future work should focus on better integrating mid-sentence lexical information.
Summary updated