Introduction
In Natural Language Generation (NLG), Referring Expression Generation (REG) is the task of selecting attributes to identify a target object among distractors. To develop human-like algorithms, researchers require high-quality corpora of natural human speech. While datasets like TUNA (English) and D-TUNA (Dutch) exist, they often lack spontaneous speech data, acoustic timing, or considerations of cognitive load.
G-TUNA addresses these gaps by providing a German-language corpus of spoken referring expressions (REs) captured in a dual-task environment. By recording participants in a driving simulator, researchers can study how speech adapts to the listener's cognitive demands and time pressure.
Experimental Setup and Methods
The G-TUNA corpus was collected using a referential communication task within a driving simulator. Participants worked in pairs: a "speaker" and a "driver/listener."
The speaker was presented with a grid of furniture images and tasked with describing a specific target so the driver could identify it. To manipulate cognitive load, the driver performed two tasks:
- EASY: Holding the steering wheel stationary.
- HARD: Steering to follow a moving lead vehicle.
All interactions were recorded, transcribed, and crucially, annotated with exact durations for each RE. This allows for a unique analysis of speech rate alongside semantic content.
The G-TUNA Corpus
G-TUNA consists of 2,331 descriptions from 40 native German speakers. It was designed to be directly comparable to the original TUNA and D-TUNA corpora, focusing on the furniture domain to ensure high-quality semantic annotation.
As shown in the comparison table, G-TUNA differs from its predecessors by focusing on spoken modality in a high-pressure communicative situation, providing a more ecologically valid dataset for real-world applications.
Key Insights: Cognitive Load and Experience
A significant finding from the G-TUNA study is the role of "experience" in communicative adaptation. Speakers only adjusted their descriptions based on the listener's cognitive load if they had previously acted as the driver. Those who had experienced the "HARD" condition themselves were more likely to produce shorter, more efficient descriptions when speaking to a busy driver.
In terms of utterance length, G-TUNA occupies a middle ground. The expressions are longer than the written TUNA data but shorter than the spoken D-TUNA data. This suggests that while speech is naturally more wordy than writing, the time pressure of the driving task encourages relative brevity.
Results and Implications
The analysis revealed several key characteristics of human communication in this context:
- Overspecification: Despite the time pressure, humans provided more information than strictly necessary in 45.5% of cases, confirming that overspecification is a robust cross-linguistic trait.
- Task Influence: "Experienced" speakers reduced their utterance length from 6.3 words (2687 ms) in the EASY condition to 5.9 words (2464 ms) in the HARD condition.
- Efficiency: Only 1.9% of expressions were underspecified, indicating that participants prioritized communicative success even under load.
Limitations and Future Work
While G-TUNA provides a rich dataset for German REG, it is currently limited to the furniture domain. Future work aims to expand this to other domains, such as "people," which typically involve more complex linguistic variations. Additionally, the researchers plan to use the duration data to evaluate REG algorithms' ability to predict not just attribute selection, but also speech rate and timing based on the listener's state.
Summary updated