Abstract
Voice assistants often struggle with non-English languages and specific demographics like the elderly due to a lack of representative training data. This study presents a multi-stage pipeline that leverages Large Language Models (LLMs) to generate synthetic German training data for intent recognition. By training efficient, small-scale transformer models (BERT, DistilBERT, Electra) on text generated by a domain-specific German LLM (LeoLM), the researchers achieved up to 83.01% accuracy on real-world elderly speech. This approach demonstrates that generative AI can effectively bridge data gaps for low-resource languages and specialized user groups while maintaining privacy through local model execution.
Introduction
Modern voice assistants are predominantly optimized for English and rigid, keyword-based commands. This creates significant accessibility barriers for elderly or handicapped users who favor natural, conversational interactions. A primary obstacle to improving these systems is the "data desert" for non-English languages and senior demographics, whose speech patterns differ significantly from the younger populations typically represented in training sets. Collecting real-world audio from these groups is costly and raises privacy concerns. This research investigates whether LLMs can synthesize high-quality training data to create more inclusive and robust voice assistants.
Methodology
The researchers developed a pipeline to build a robust Intent Recognition (IR) system without requiring massive real-world datasets for initial training.
- Synthetic Data Generation: Three LLMs—LeoLM-13b (German-specialized), Llama3-8b, and ChatGPT—were used to generate approximately 2,500 diverse German sentences across six intent classes (e.g., "help", "roll_up", "no_command").
- Speech Recognition (ASR): A Whisper-small model was fine-tuned on elderly German speech using "Experience Replay" to adapt to the domain without losing general linguistic knowledge.
- Intent Classification: Small, efficient transformer models (BERT, DistilBERT, and Electra) were trained on the synthetic text. These models are suitable for real-time, local deployment.
- Evaluation: To simulate real-world performance, synthetic text was converted back to audio using the XTTS-v2 text-to-speech model and tested against the SVC-de (Senior Voice Commands - German) dataset.
Key Insights
A critical finding was that domain expertise in LLMs outweighs raw parameter count. LeoLM-13b, despite being smaller than ChatGPT, produced higher-quality German training data because of its specific pre-training on German text. This led to better performance in the downstream classification models.
Furthermore, the researchers addressed the "no command" trigger problem—where assistants activate accidentally—by using few-shot prompting to generate realistic "false positive" sentences, such as casual conversation not intended as a command.
Results and Discussion
The combination of Whisper and DistilBERT, when trained on LeoLM-generated data, achieved 83.01% accuracy on the real-world SVC-de dataset. When evaluated on synthetic test sets, accuracy reached as high as 96.98%.
The synthetic speech used for testing exhibited a Word Error Rate (WER) of 9.5% to 15%, closely matching the ~11% WER observed in real-world elderly speech. This validates the use of synthetic audio for robust system evaluation. Crucially, the final models are small enough to run locally, ensuring user privacy by processing data within the home.
Conclusion
This research demonstrates a viable path for developing specialized voice assistants in low-resource contexts. By using LLMs as "data hallucination" engines, developers can overcome the scarcity of demographic-specific data. Future work will explore expanding the intent library to multi-step commands and testing the pipeline with speakers having diverse speech impairments or regional dialects.
Summary updated