TL;DR
This study investigates whether real-time visual feedback of tongue movement using Electromagnetic Articulography (EMA) can improve the acquisition of a non-native speech sound (the Japanese flap) in adult English speakers. The findings suggest that EMA-based feedback leads to better acquisition and retention of the target sound compared to traditional speech training methods.
What Problem Does This Paper Address?
Learning to produce non-native speech sounds can be challenging for adults. While visual feedback has been explored in speech therapy for individuals with communication disorders, its effectiveness in teaching new sounds to healthy adults acquiring a second language (L2) is less understood. Specifically, this paper addresses how augmented visual feedback, which directly displays articulator movements, might enhance L2 phoneme acquisition compared to conventional training methods. The Japanese flap /t/ is chosen as the target sound because it is phonetically distinct from English sounds and often difficult for English speakers to produce correctly.
Methods
The study employed a single-subject ABA design with eight adult native English speakers. Participants were randomly assigned to one of two training conditions:
- Conventional L2 Speech Production Training (non-EMA): Participants received standard instruction and auditory models.
- Conventional L2 Speech Production Training with EMA-based Kinematic Feedback (EMA): In addition to standard training, participants received real-time visual feedback of their tongue tip position using Electromagnetic Articulography (EMA). EMA uses small sensors attached to articulators to track their movements in a mid-sagittal plane, displayed on a computer screen.
Participants practiced producing 16 disyllabic words containing the Japanese flap. Speech productions were recorded during baseline, training (eight sessions), and post-training (four weeks later) phases. Perceptual judgments were made by six native Japanese speakers who rated whether the produced sound was a "Japanese flap" or "not a Japanese flap." Effect sizes were calculated using Percentage of Non-overlapping Data (PND).
Datasets
The study did not introduce new datasets for training or evaluation. Instead, it used probe data collected from the participants' speech productions during the experiment. The stimuli consisted of 16 disyllabic words, some of which were meaningful in Japanese, but were used as non-words for the training purpose.
Key Insights and Ideas
- Visual Feedback Enhances L2 Learning: The core insight is that real-time visual feedback of articulatory movements can significantly aid in learning a difficult non-native speech sound.
- EMA Facilitates Acquisition and Retention: The EMA group showed not only better acquisition of the Japanese flap but also improved retention of the sound four weeks after training concluded.
- Different Learning Trajectories: The non-EMA group showed more immediate improvement early in training, but this plateaued. In contrast, the EMA group demonstrated a more gradual, continuous improvement throughout the training period.
- Auditory/Visual Integration Takes Time: The authors hypothesize that the initial delay in EMA benefits might be due to the time required for participants to integrate visual and auditory information and coordinate their eye-tongue movements.
Key Results
- Perceptual Judgments: Native Japanese listeners rated the productions of participants in the EMA condition as more frequently being a "Japanese flap" compared to the non-EMA condition, especially in the post-training retention phase.
- Word-Initial Flaps: Figure 1 shows noticeably higher retention levels for word-initial flaps in the EMA condition.
- Word-Medial Flaps: For word-medial flaps (Figure 2), both conditions showed improvement, but the EMA condition still suggested better performance.
- Effect Size (PND):
- Baseline vs. First Half of Training (T1-4): EMA mean PND was 46%, non-EMA was 65%. This suggests a more immediate effect in the non-EMA group initially, but neither reached "fairly effective" levels.
- Baseline vs. Second Half of Training (T5-8): EMA mean PND was 80% ("fairly effective" to "highly effective"), non-EMA was 66% ("questionable effectiveness" to "highly effective"). This indicates EMA's effectiveness increased while non-EMA plateaued.
- Baseline vs. Post-training (P1-2): EMA mean PND was 80% ("questionable effectiveness" to "highly effective"), non-EMA was 67% ("questionable effectiveness" to "highly effective"). This highlights better retention in the EMA group.
Limitations and Future Work
- Limitations:
- Not Double-Blind: The experimental design was not double-blind, meaning instructor bias could have influenced results.
- ABA Design: The ABA design did not allow for the exploration of skill transfer and generalization.
- No Mastery Criteria: Participants were not trained to mastery, so it's unknown if they could reach native-like production with further training.
- Future Work: The authors suggest future studies should examine varied feedback presentation schedules, different types of motor targets, and other factors influencing motor learning to further understand augmented visual feedback's role in L2 acquisition. They also call for replication of the study to ensure validity.
Summary updated