Modern generative audio models excel at creating short, high-fidelity snippets of sound. Whether using diffusion models or autoregressive transformers, generating a convincing 5-to-30-second clip of a specific genre or instrument is largely a solved problem. However, as noted by Yixiao Zhang et al. in their work on Instruct-MusicGen, scaling these systems to maintain structural coherence over the duration of a full-length song remains a major hurdle.
When forced to generate multi-minute audio, standard models suffer from "structural amnesia": they drift aimlessly between keys, lose track of the underlying rhythm, and fail to produce macro-level architectural forms like verse-chorus-bridge transitions. Achieving long-form structural coherence requires moving past local audio synthesis toward global architectural planning.
The Root of Structural Amnesia
Why do models that are so effective at short-term synthesis fail over longer horizons? The issue stems from how audio is traditionally modeled. Most deep learning architectures treat audio as a flat sequence of local features or discrete tokens. While sequential models are excellent at predicting the immediate next frame or note based on local context, Alexey Tikhonov et al. observe that they frequently fail to maintain the overarching macro-structure and stylistic consistency required for coherent musical genres.
In autoregressive models, small errors accumulate over time, causing the generation to drift away from the initial thematic intent. In diffusion models, as Jan Melechovský et al. point out in Mustango, optimizations are typically bounded by short temporal windows, leaving full-length composition an open problem. Without an explicit global plan, the model has no memory of whether it is in an introduction, a verse, or a conclusion—concepts that Flavio Schneider et al. emphasize are vital for complete musical works in Moûsai.
Hierarchical Representations: Separating Skeleton from Detail
To prevent models from drifting, researchers are moving away from flat token sequences toward hierarchical representations. The intuition is simple: human composers do not write a symphony note-by-note from left to right; they sketch a high-level structural outline before filling in the fine-grained acoustic details.
A prominent example of this is PyramidCodec, introduced by Jianyi Chen et al.. This hierarchical audio tokenizer represents music across multiple temporal scales simultaneously. By separating a compact structural "skeleton" from high-fidelity acoustic details, it allows transformers to maintain long-term rhythmic and thematic consistency over several minutes.
Similarly, in the symbolic domain, the authors of Modeling Temporal Tonal Relations in Polyphonic Music Through Deep Networks With a Novel Image-Based Representation utilize a 2D "Tonnetz" grid to encode harmonic relationships directly into the data structure. This provides the inductive bias necessary to maintain tonal stability and structural repetition over time, preventing the aimless or disorganized outputs common in unconstrained sequence models.
Inference-Time Continuity and Seamless Outpainting
Another major challenge in long-form generation is the emergence of audible "seams" or abrupt stylistic shifts when models concatenate shorter audio blocks. Traditional feature concatenation often struggles with computational efficiency and fails to prevent drifting.
To combat this, Chaeyoung Jung et al. developed InfiniteAudio, a training-free inference strategy that uses a sliding-window "FIFO" sampling approach combined with a "curved denoising" technique. By prioritizing critical diffusion steps across overlapping windows, it eliminates boundary artifacts while maintaining a constant memory footprint.
In a similar vein, Hounsu Kim et al. leverage diffusion outpainting alongside a dense, instrument-specific "guitarroll" representation. This allows the model to generate extended instrumental performances by treating subsequent segments as natural extensions of the existing context, ensuring superior timbre and long-term consistency.
Structural Scaffolding via Task Decomposition and Explicit Conditioning
Instead of expecting a model to implicitly discover long-range dependencies, a third paradigm enforces macro-structure through explicit conditioning and task decomposition.
For instance, AudioStory, a framework by Yuxin Guo et al., breaks down complex, long-form prompts into structured sub-tasks. By utilizing a "decoupled bridging" mechanism that handles semantic and residual tokens separately, the system maintains a consistent emotional tone and acoustic atmosphere across sequences lasting up to 150 seconds.
Explicitly injecting technical musical features also serves as an effective anchor. Abhinaba Roy et al. demonstrate in SonicVerse that predicting and injecting "hard" musical features—such as key, instrumentation, and vocal presence—as explicit tokens into a Large Language Model allows the system to track the temporal evolution of full-length musical works, successfully overcoming the limited context windows of standard audio encoders.
The Evaluation Shift
Progress in long-form structural coherence has historically been bottlenecked by evaluation metrics. As Florian Grötschla et al. point out in Benchmarking Music Generation Models and Metrics via Human Preference Studies, standard quantitative metrics are heavily optimized for short-duration clips, failing to capture macro-level properties like thematic development and structural progression. High performance on short-form benchmarks rarely translates to the production of coherent full-length compositions.
To bridge this gap, datasets like SongEval, introduced by Jixun Yao et al., provide full-length audio tracks annotated across multi-minute dimensions. This allows researchers to accurately benchmark how well a model transitions between distinct structural sections like intros, verses, and choruses, shifting the focus of the field from local acoustic fidelity to global architectural integrity.