TL;DR
This paper presents a task-independent Japanese text input system that uses speech recognition as its primary input method, augmented by a keyboard. The system tackles the complexity of Japanese text by focusing on robust phoneme discrimination using an articulatory model and then employing a sophisticated word discrimination process that leverages a vowel-indexed dictionary and dynamic programming.
What Problem Does This Paper Address?
Inputting Japanese text into computers is a significant challenge due to the language's complex writing system, which uses a combination of Kanji (ideographic characters), Hiragana (cursive syllabary), Katakana (square syllabary), and special symbols. Traditional keyboard input for Japanese can be cumbersome, requiring multiple keystrokes or complex input methods. The authors aim to create a more natural and efficient input method by using speech recognition, but this introduces its own set of difficulties, particularly in accurately recognizing phonemes and then translating those phonemes into the correct words and characters. The core problem is building a speech recognition system that is not tied to a specific task (task-independent) and can handle the nuances of Japanese text input.
Methods
The system is divided into two main parts: phoneme discrimination and word discrimination.
- Phoneme Discrimination: This part focuses on segmenting continuous speech into discrete phonemic strings. A novel approach is employed using an articulatory motion estimation method based on a speech production model. This method estimates the articulatory movements (like tongue and lip positions) from speech waves, which is believed to better handle coarticulation and speaker variability than traditional acoustic feature extraction. The outputs are lattices of pseudo-phonemes.
- Word Discrimination: This part takes the phonemic strings and corrects them using various forms of prior knowledge to transform them into suitable characters. Key techniques include:
- Vowel-Indexed Word Dictionary Retrieval: The dictionary is organized by vowels, leveraging the fact that vowels are more reliably discriminated than consonants.
- Dynamic Programming (DP) for Similarity Calculation: DP is used to calculate the similarity between the recognized phonemic strings and entries in the word dictionary, incorporating a phonemic confusion matrix.
- Articulatory Model: A detailed articulatory model is used, representing the vocal tract shape and movement with a set of parameters. This model is then used to estimate articulatory motion from speech.
- Speaker Normalization: Techniques are developed to normalize articulatory parameters, making the system more speaker-independent, including a linear transformation to align female speaker data with male speaker data.
- Consonant Recognition: For consonants, which are more transient, a combination of DP matching on articulatory parameters and acoustic parameters is used, particularly focusing on the gliding sections between consonants and vowels.
- Keyboard Augmentation: A simple 7-key keyboard is used to provide control information, correct phonemic data, and assist in Kana-Kanji translation, especially for ambiguous cases.
Datasets
The paper describes experiments using speech data from adult male and female speakers. Specific details on the size and nature of these datasets are provided:
- Vowel Recognition Experiments: Involved 50 subjects (30 males, 20 females), with each subject uttering each vowel five times. Data was grouped into male groups (A, B, C) and female groups (a, b).
- Consonant Recognition Experiments: Used speech data from 2 male adults, each speaking continuous V1CV2 sequences (where V1=/a/, V2 is a vowel, and C is a consonant).
- Continuous Vowel Discrimination: Involved 2 male adults, each speaking 23 kinds of continuous 5-vowel sequences twice.
The paper does not explicitly mention the use of publicly available benchmark datasets.
Key Insights and Ideas
- Articulatory Model for Robust Phoneme Recognition: The core idea is that modeling the physical movements of speech production (articulatory motion) provides a more robust feature set for phoneme discrimination, especially for handling coarticulation and speaker variability, compared to purely acoustic features.
- Vowel-Centric Word Discrimination: Recognizing that vowels are more reliably discriminated than consonants, the system leverages this by indexing its word dictionary by vowels, significantly improving the efficiency and accuracy of word retrieval.
- Dynamic Programming with Confusion Matrices: The use of DP with a phonemic confusion matrix is a standard but effective technique for sequence matching, and its application here, informed by the specific characteristics of their phoneme recognizer, is crucial for word discrimination.
- Speaker Normalization is Key for Task-Independence: The development of methods to normalize articulatory parameters, particularly the linear transformation to bridge male-female differences, is essential for achieving a truly task-independent system.
- Hybrid Input for Practicality: The integration of a simple keyboard as an auxiliary input acknowledges the inherent difficulties in perfect speech recognition and provides a practical mechanism for correction and disambiguation, especially for complex tasks like Kana-Kanji translation.
Key Results
- Vowel Recognition: The paper reports high accuracy in vowel recognition experiments, with results like 98.8% for male voices after learning by male voices (Experiment I) and 97.3% for female voices using normalization of linear transformation (Experiment IV).
- Consonant Recognition: Consonant recognition results are presented in Table 2-4, showing varying accuracy depending on the consonant type and whether it's voiced or unvoiced. For voiced consonants, DP matching of articulatory parameters was reliable. For unvoiced consonants, acoustic parameters were mainly used. Overall accuracy for voiced consonants was around 91.4% and for unvoiced consonants around 85.4%.
- Normalization Effectiveness: The normalization procedure for articulatory parameters was shown to be effective for male-female transformation, improving recognition rates.
Limitations and Future Work
- Consonant Discrimination Challenges: The paper acknowledges that while the articulatory motion feature extraction for consonants shows promise, there are limitations in precision and stability, and it is influenced by the speaker. A combination with other sound parameters is suggested for improvement.
- Syntax in Word Discrimination: The current version of the word discrimination part does not extensively use syntax, except for Hiragana. Future work plans to incorporate syntactic information to improve performance.
- Trade-off Between Facility and Speed: The authors note that the trade-off between system facility (ease of use) and processing speed is an important consideration for future development.
- Ambiguity in Phoneme Strings: The system relies on a keyboard for correction, indicating that perfect phoneme string generation is still a challenge.
- Limited Dictionary Size: The Kanji dictionary used in the word discrimination part contains about 1000 readings. Expanding this dictionary would be a natural area for future work.
Summary updated