Abstract
In this paper, we introduce Bardo, a real-time intelligent system designed to automatically select background music for tabletop role-playing games (TTRPGs). Bardo utilizes off-the-shelf speech recognition to transcribe player dialogue and a supervised learning algorithm to classify the emotional state of the scene into one of four categories: Happy, Calm, Agitated, or Suspenseful. Based on this classification, the system dynamically selects and plays appropriate background music. Evaluation using a Dungeons & Dragons (D&D) campaign demonstrates that a Naive Bayes classifier with a sliding window approach achieves high prediction accuracy. Furthermore, a user study indicates that Bardo’s automated selections are often preferred over the original human-curated music.
Introduction: The AI Dungeon Master’s Personal DJ
In TTRPGs like Dungeons & Dragons, immersion is a primary goal. While background music significantly enhances the atmosphere, managing it manually is a significant burden for Dungeon Masters (DMs), who must already manage complex rules, non-player characters, and narrative flow. Automating this process is challenging due to the "messy" nature of live play, characterized by ungrammatical speech, overlapping dialogue, and rapid emotional shifts. Bardo addresses these challenges by providing a real-time, context-aware music recommendation system.
Methodology
The Bardo pipeline consists of three main stages:
- Speech-to-Text: The system captures player dialogue using an off-the-shelf speech recognition system. To maintain robustness against transcription errors, the text is processed as a "bag of words," prioritizing keywords over complex syntax.
- Emotion Classification: We compared lexicon-based approaches (Density-Based) with probabilistic models (Naive Bayes). The system classifies dialogue into four emotional states: Happy, Calm, Agitated, and Suspenseful.
- FSM Controller: To ensure smooth musical transitions and prevent rapid "jittering," Bardo employs a Finite State Machine (FSM). This controller uses a sliding window to incorporate temporal context and a confidence threshold to trigger music changes.
Dataset and Annotation
The system was trained and evaluated on a custom dataset derived from the YouTube D&D campaign Call of the Wild. The dataset comprises 5,892 sentences (approximately 4.5 hours of gameplay) labeled by three human annotators. This dataset highlights the unique linguistic patterns of TTRPG players, where specific "game-talk" (e.g., "initiative," "stealth") serves as a strong indicator of emotional state.
Key Results
Classification Accuracy
Experimental results showed that the Naive Bayes variant utilizing a sliding window (NS) was the most effective model. It achieved an average accuracy of 64% across nine episodes, which is significant given the inherent noise in transcribed live speech.
User Preference
A user study involving 61 participants compared Bardo’s music selections against the original human-edited audio from the Call of the Wild videos. Participants preferred Bardo’s selections 163 times, compared to 74 for the baseline. Notably, in some instances, Bardo’s automated choices were deemed more appropriate than the original human labels upon retrospective review.
Discussion and Limitations
While Bardo demonstrates the feasibility of automated music selection for TTRPGs, several limitations remain. The "Happy" emotion was underrepresented in the training data, making it difficult for the model to learn. Additionally, the current system relies solely on text, ignoring acoustic features like tone and prosody. Future work will focus on integrating multi-modal data and testing the system in live, face-to-face environments with higher ambient noise and overlapping speech.
Summary updated