Large Language Model Data Generation for Enhanced Intent Recognition in German Speech

Theresa Pekarek-Rosin, Burak Can Kaplan, Stefan WermterarXiv 2025 · DOI · arXiv

Abstract

Voice assistants often struggle with non-English languages and specific demographics like the elderly due to a lack of representative training data. This study presents a multi-stage pipeline that leverages Large Language Models (LLMs) to generate synthetic German training data for intent recognition. By training efficient, small-scale transformer models (BERT, DistilBERT, Electra) on text generated by a domain-specific German LLM (LeoLM), the researchers achieved up to 83.01% accuracy on real-world elderly speech. This approach demonstrates that generative AI can effectively bridge data gaps for low-resource languages and specialized user groups while maintaining privacy through local model execution.

Introduction

Modern voice assistants are predominantly optimized for English and rigid, keyword-based commands. This creates significant accessibility barriers for elderly or handicapped users who favor natural, conversational interactions. A primary obstacle to improving these systems is the "data desert" for non-English languages and senior demographics, whose speech patterns differ significantly from the younger populations typically represented in training sets. Collecting real-world audio from these groups is costly and raises privacy concerns. This research investigates whether LLMs can synthesize high-quality training data to create more inclusive and robust voice assistants.

Methodology

The researchers developed a pipeline to build a robust Intent Recognition (IR) system without requiring massive real-world datasets for initial training.

  1. Synthetic Data Generation: Three LLMs—LeoLM-13b (German-specialized), Llama3-8b, and ChatGPT—were used to generate approximately 2,500 diverse German sentences across six intent classes (e.g., "help", "roll_up", "no_command").
  2. Speech Recognition (ASR): A Whisper-small model was fine-tuned on elderly German speech using "Experience Replay" to adapt to the domain without losing general linguistic knowledge.
  3. Intent Classification: Small, efficient transformer models (BERT, DistilBERT, and Electra) were trained on the synthetic text. These models are suitable for real-time, local deployment.
  4. Evaluation: To simulate real-world performance, synthetic text was converted back to audio using the XTTS-v2 text-to-speech model and tested against the SVC-de (Senior Voice Commands - German) dataset.
The multi-stage pipeline for intent recognition, showing the flow from speech input through Whisper ASR to small transformer classifiers.

Key Insights

A critical finding was that domain expertise in LLMs outweighs raw parameter count. LeoLM-13b, despite being smaller than ChatGPT, produced higher-quality German training data because of its specific pre-training on German text. This led to better performance in the downstream classification models.

Furthermore, the researchers addressed the "no command" trigger problem—where assistants activate accidentally—by using few-shot prompting to generate realistic "false positive" sentences, such as casual conversation not intended as a command.

Results and Discussion

The combination of Whisper and DistilBERT, when trained on LeoLM-generated data, achieved 83.01% accuracy on the real-world SVC-de dataset. When evaluated on synthetic test sets, accuracy reached as high as 96.98%.

Confusion matrices comparing the performance of BERT, DistilBERT, and Electra models trained on data from different LLMs (ChatGPT, LeoLM, Llama3).

The synthetic speech used for testing exhibited a Word Error Rate (WER) of 9.5% to 15%, closely matching the ~11% WER observed in real-world elderly speech. This validates the use of synthetic audio for robust system evaluation. Crucially, the final models are small enough to run locally, ensuring user privacy by processing data within the home.

Conclusion

This research demonstrates a viable path for developing specialized voice assistants in low-resource contexts. By using LLMs as "data hallucination" engines, developers can overcome the scarcity of demographic-specific data. Future work will explore expanding the intent library to multi-step commands and testing the pipeline with speakers having diverse speech impairments or regional dialects.

Summary updated

Abstract

Intent recognition (IR) for speech commands is essential for artificial intelligence (AI) assistant systems; however, most existing approaches are limited to short commands and are predominantly developed for English. This paper addresses these limitations by focusing on IR from speech by elderly German speakers. We propose a novel approach that combines an adapted Whisper ASR model, fine-tuned on elderly German speech (SVC-de), with Transformer-based language models trained on synthetic text datasets generated by three well-known large language models (LLMs): LeoLM, Llama3, and ChatGPT. To evaluate the robustness of our approach, we generate synthetic speech with a text-to-speech model and conduct extensive cross-dataset testing. Our results show that synthetic LLM-generated data significantly boosts classification performance and robustness to different speaking styles and unseen vocabulary. Notably, we find that LeoLM, a smaller, domain-specific 13B LLM, surpasses the much larger ChatGPT (175B) in dataset quality for German intent recognition. Our approach demonstrates that generative AI can effectively bridge data gaps in low-resource domains. We provide detailed documentation of our data generation and training process to ensure transparency and reproducibility.
BibTeX
@misc{pekarekrosin2025large,
  author = {Theresa Pekarek-Rosin and Burak Can Kaplan and Stefan Wermter},
  title = {{Large Language Model Data Generation for Enhanced Intent Recognition in German Speech}},
  year = {2025},
  eprint = {2508.06277},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2508.06277},
}
Figure