Abstract
My main research motivation is to develop complete autonomous agents that interact with people socially. For an agent to be social with respect to humans, it needs to be able to parse and process the multitude of aspects that comprise the human cultural experience. Music, as a general target domain, serves as an excellent testbed for these research ideas. Musical skills—playing music, analyzing music, or composing it—all involve extremely advanced knowledge representation and problem-solving tools. Creating "musical agents" that can interact richly with people is a challenge that holds the potential of advancing social agents research and contributing important AI knowledge. This thesis focuses on two aspects of musical intelligence: music recommendation and multiagent interaction in the context of music, guided by a sequential decision-making perspective.
Introduction
Music is rarely experienced in a vacuum. We do not simply listen to a song; we listen to a sequence of songs, often while performing tasks that the music subtly influences. Traditional Artificial Intelligence (AI) approaches to music have largely focused on static tasks, such as genre classification or one-off recommendations. However, for AI to truly achieve "musical intelligence," it must move beyond simple labeling and embrace sequential decision-making. This research argues that by applying Reinforcement Learning (RL) and cognitive modeling, we can develop agents that build better playlists, adapt to shifting human tastes, and eventually collaborate in shared creative spaces.
The Sequential Nature of Music Recommendation
Most traditional music AI systems treat songs as independent data points. If a user likes Song A, the system recommends Song B based on similarity. This approach ignores the critical temporal context of the listening experience. For example, a high-energy workout track might be appropriate at 8:00 AM but jarring if placed immediately after a lo-fi jazz piece.
To address this, the research utilizes Reinforcement Learning (RL) to shift away from "greedy" recommendation—picking the single best next song—toward optimizing an entire sequence. By framing playlist generation as a Markov Decision Process, the agent (referred to as DJ-MC) learns a model of both individual song preferences and the transition preferences between songs. To solve the "cold start" problem inherent in RL, an active initialization algorithm is used to gather direct user feedback, significantly reducing the time the agent spends in suboptimal exploration phases.
Music as a Cognitive Primer
A core component of this research involves understanding how musical stimuli affect human cognitive processes. Using the Drift-Diffusion Model (DDM)—a stochastic sequential model—the research analyzes how music influences decision-making. The DDM allows for the decomposition of observed behaviors into underlying components, such as boundary separation (caution) and drift rate (information processing speed).
Experimental results indicate that music does more than just change a listener's mood; it alters their logic and decision-making efficiency. Participants listening to "happy" music demonstrated increased speed and quality in quantitative tasks, such as gambling ratios and word classification. Furthermore, a direct correlation was noted between the tempo of the musical stimulus and the speed of the human's decision-making process. This suggests that a truly intelligent musical agent must account for the cognitive "nudge" its choices provide to the listener.
Social Interaction and Multiagent Systems
Beyond individual recommendation, the research explores the social aspects of music through multiagent simulation and joint improvisation. Human musical taste is a moving target, shifting with fashion, mood, and time—a phenomenon known as "concept drift." The research proposes framing "model management" itself as an RL problem, where the agent decides when and how to adapt its internal models to mitigate the risks of changing user data.
In the context of joint improvisation, the challenge is for agents to collaborate in a shared creative space without knowing each other's internal reward functions. To be a truly social agent, the AI must model the intentions of its human counterparts. The agent's actions are dual-purpose: they must satisfy an aesthetic goal while simultaneously accounting for how those actions affect the human's perception and subsequent musical choices.
Conclusion
By shifting the perspective of musical AI from static classification to sequential decision-making, this research provides a framework for more sophisticated social agents. The results from the DJ-MC agent and cognitive priming studies demonstrate that considering the temporal and psychological context of music leads to higher user enjoyment and more effective human-agent interaction. While the collaborative aspects of this work remain ongoing, the ultimate goal is to move AI from being a tool we use to listen to music, to a social entity we listen with.
Summary updated