A 3-d virtual head as a tool for speech therapy for children

Sascha Fagel, Katja MadanyInterspeech 2008

TL;DR

This paper introduces "Vivian," a 3D virtual talking head designed to aid speech therapists in treating children with phonetic disorders, specifically focusing on the /s/ and /z/ sounds. An initial evaluation showed that Vivian significantly improved speech production for most participating children, demonstrating its potential as an engaging and effective therapeutic tool.

What Problem Does This Paper Address?

Human speech is both audible and visible, and the visual component can significantly enhance intelligibility. Traditional speech therapy often relies on two-dimensional diagrams or physical demonstrations, which can be limited in their ability to convey the complex, internal movements of the articulators (tongue, jaw, velum). This paper addresses the challenge of providing a more intuitive and engaging way for children to understand and practice correct speech sound production, particularly for sounds like /s/ and /z/ which involve subtle tongue positioning. The goal is to create a tool that can visually explain articulatory processes in 3D without invasive methods, making therapy more effective and accessible.

Methods

The core of the system is "Vivian," a software tool built using HTML and JavaScript, making it platform-independent and runnable from a CD-ROM. It requires a web browser with a VRML plugin for 3D animation. Vivian utilizes a 3D virtual head model, created with Digimation Head Designer and 3ds Max, with manually designed internal articulators based on MRI data.

The system generates audiovisual speech examples using the MASSY (Modular Audiovisual Speech SYnthesizer). MASSY takes text input, converts it into phonetic transcriptions with prosody, synthesizes the audio using the MBROLA engine, and generates visual articulation parameters for the virtual head. These parameters control movements of the lips, jaw, tongue, and velum.

The graphical user interface (GUI) is designed for therapists, allowing them to select specific German consonants and affricates, and then choose words containing these sounds in various positions (isolated, initial, medial, final, or in clusters). Therapists can also adjust display settings like skin transparency, zoom level, orientation, and playback speed.

For the initial evaluation, an experimenter used a predecessor version of Vivian to teach children correct /s/ and /z/ production. The learning lesson involved showing the virtual head's articulation with transparent skin, demonstrating the correct tongue placement behind the upper incisors.

Datasets

The paper does not explicitly mention the use of pre-existing datasets for training the speech synthesis or animation models. However, the training material for the speech therapy itself was selected based on standard diagnostic literature for children's speech disorders, including PLAKSS, AVAK, and Patholinguistic Diagnostics for Disorders of Speech Development. Simple nouns were chosen for their familiarity and potential for future picture-based visualization. The paper states that all German consonants and their phonotactically possible combinations were included in an isolated form.

Key Insights and Ideas

  • 3D Internal Articulation Visualization: The ability to display the internal articulators (tongue, jaw, velum) transparently within a virtual head offers a novel way to explain complex speech movements that are difficult to convey with 2D diagrams or external observation.
  • Engaging Therapeutic Tool: The virtual head, particularly with its interactive and visual nature, appears to be highly engaging for children, potentially increasing their motivation and participation in therapy.
  • Potential for Broader Application: While focused on /s/ and /z/, the system's design suggests it could be extended to visualize the production of a wider range of speech sounds.
  • Positive Impact on Speech Production: The initial evaluation indicates that the visual feedback provided by the virtual head can lead to significant improvements in children's ability to produce target sounds.

Key Results

The initial evaluation involved eight children aged 4;11 to 7;11. After one learning lesson with Vivian, six out of the eight children showed a significant decrease in their mean degree of lisping. The overall mean degree of lisping across all children (excluding one who didn't complete all tests) decreased from 2.33 before the first lesson to 1.85 after the first lesson and 1.83 after the second. One child showed no significant improvement, which was attributed to potential myofunctional disorders. Another child showed a slight increase in lisping after the second lesson but still a significant improvement from the initial score. Informal interviews revealed that children were fascinated by the talking head and expressed interest in learning other sounds.

Limitations and Future Work

Limitations: * The study involved a small sample size (eight children), limiting the generalizability of the findings. * The evaluation was conducted with only one specific speech disorder (sigmatismus interdentalis) and focused on only two sounds (/s/, /z/). * One child's lack of improvement was attributed to potential myofunctional disorders, suggesting that Vivian may not be a universal solution for all speech impediments. * The paper acknowledges that while children seemed to interpret the internal articulator displays quickly, direct comparisons to other visual feedback methods (like a normal synthetic head) did not yield clear evidence for the superiority of internal articulation display.

Future Work: * Further evaluation will be conducted with a larger number of children, a wider range of speech disorders, and a control group not using the software. * The authors plan to extend the system with picture-based visualization for a more comprehensive therapeutic experience. * The potential for a simple practice effect being unlikely due to the short duration of the learning lessons is noted, but further investigation into the learning mechanisms could be beneficial.

Summary updated

BibTeX
@inproceedings{fagel2008d,
  author = {Sascha Fagel and Katja Madany},
  title = {{A 3-d virtual head as a tool for speech therapy for children}},
  booktitle = {Interspeech},
  year = {2008},
}
Figure