Improving AI-generated music with user-guided training

Vishwa Mohan Singh, Sai Anirudh Aryasomayajula, Ahan Chatterjee, Beste Aydemir, Rifat Mehreen AminarXiv 2025 · DOI · arXiv

Abstract

Generative AI for music often struggles to capture the subjective nuances of human taste through traditional text-based prompting. This paper introduces a human-in-the-loop framework that personalizes music generation by treating user ratings and listening behavior as a dynamic loss function. By iteratively fine-tuning a latent diffusion model based on human computation (HC), the system moves beyond static datasets to create audio that better aligns with individual and community preferences. Experimental results demonstrate a significant increase in user satisfaction over multiple training iterations.

Introduction

While generative models like Riffusion and Jukebox have demonstrated impressive technical capabilities, they frequently encounter a "subjectivity wall." Text prompts serve as a low-bandwidth medium for describing complex musical elements; it is difficult to articulate the precise "feel" of a bassline or the subtle blending of disparate genres through words alone.

Furthermore, music is inherently subjective. A composition that resonates with one listener may be perceived as noise by another. Traditional models trained on fixed datasets lack the adaptive mechanisms required to evolve with user preferences. To bridge this gap, we propose a system that integrates human feedback directly into the training loop, allowing the model to learn from explicit ratings and implicit behavioral cues.

System Architecture and Design

The proposed framework is divided into four primary modules: the User Interface, System Logic, Database, and Music Generator. This structure facilitates a continuous cycle of generation, evaluation, and refinement.

Figure 5: The comprehensive system architecture, illustrating the flow between the user interface, dispatching logic, and the music generation modules.

As shown in the architecture, the Music Dispatching Logic manages the distribution of songs to users, while the Training Pair Creation module prepares data for fine-tuning. The system utilizes a "Music Roll Out" algorithm to balance exploration and exploitation: 25% of the time, it presents high-scoring baseline songs, while 75% of the time, it generates new content to explore the latent space and gather fresh human data.

Methodology

Technical Pillars

Our approach leverages several key technical innovations to enable high-quality, guided music generation:

  • Audio-to-Audio Conditioning: Users provide reference songs which are converted into spectrograms and latent vectors, providing a higher-fidelity starting point than text.
  • Denoising Diffusion Implicit Models (DDIM): We utilize DDIM for efficient sampling, feeding user-provided songs as intermediate latents during the denoising process.
  • Spherical Linear Interpolation (SLERP): To combine multiple input styles, the system uses SLERP to interpolate between latent noise vectors, ensuring a smooth transition between musical characteristics.
  • Human-Guided Loss Function: The core of the iterative process is a "confidence score" ($\omega$). This metric is a weighted mean of explicit user star ratings and implicit feedback, such as total listening time. This prevents "adversarial" data—where a user might rate a song highly but skip it immediately—from corrupting the model.

Training Iterations

The base model (v0) utilized pre-trained weights from the teticio/audio-diffusion-256 pipeline. In version 1 (v1), the model was fine-tuned on the top 100 songs from a 5-user pilot. Version 2 (v2) expanded this to the top 100 songs from a larger 50-user pool.

Experimental Results

A pilot study involving 50 users compared the three iterations of the model. The results indicate a clear upward trend in perceived quality as the model incorporated more human feedback.

Figure 6: Comparison of average user ratings and sample counts across three model versions (v0, v1, and v2).

The baseline model (v0) achieved an average rating of 2.75. Following the first iteration of human-guided fine-tuning, the average rating rose to 2.95 (v1). By the second iteration (v2), the rating climbed significantly to 3.34.

While the overall average improved, we observed that for certain specific input songs, earlier versions occasionally outperformed later ones. This suggests a potential "drift" where the model optimizes for majority tastes at the expense of niche characteristics. Additionally, implicit feedback served as a vital sanity check; by adjusting ratings based on listening time, the system maintained data integrity against low-effort user inputs.

Discussion and Future Work

Despite the success of the iterative framework, several challenges remain. Users reported that merging multiple songs often resulted in audio distortion and a loss of distinctiveness, leading us to exclude multi-song merging from the final evaluation phase. Furthermore, the current sample size of 50 users, while sufficient for a pilot, must be scaled to thousands to test the limits of deep learning adaptation.

Future research will focus on a "purge cycle" to remove obsolete training samples and prevent dataset bloat. We also intend to explore more sophisticated feedback metrics, such as emotional tagging and granular playback data, to further refine the model's understanding of human musical preference.

Summary updated

BibTeX
@misc{singh2025improving,
  author = {Vishwa Mohan Singh and Sai Anirudh Aryasomayajula and Ahan Chatterjee and Beste Aydemir and Rifat Mehreen Amin},
  title = {{Improving AI-generated music with user-guided training}},
  year = {2025},
  eprint = {2506.04852},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2506.04852},
}
Figure