Abstract: The paper presents a method of the music generation based on LSTM (Long Short-Term Memory), contrasts the effects of different network structures on the music generation and introduces other methods used by some researchers.
Introduction
Music composition is traditionally viewed as a uniquely human, intuitive ability. However, from a computational perspective, music is a sequence of events—notes, chords, and rhythms—governed by complex conditional probabilities. This paper explores the use of Long Short-Term Memory (LSTM) networks to automate this creative process. By treating MIDI note sequences as a time-series prediction problem, the study demonstrates how deep LSTM architectures can learn the underlying "logic" of piano music to generate new, high-quality melodies.
The primary challenge in automatic composition is capturing long-term dependencies. Traditional Recurrent Neural Networks (RNNs) struggle with the "vanishing gradient problem," where information from the distant past evaporates during training. LSTM units are designed specifically to "remember" information over long periods, making them ideal for musical structures where a note's relevance may depend on a theme introduced much earlier.
Methods and Architecture
The implemented deep learning pipeline consists of data pre-processing, a multi-layered neural network, and an iterative generation algorithm.
The Network Architecture
The model utilizes a stack of three LSTM layers, each containing 512 hidden units. To prevent overfitting, Dropout layers with a rate of 0.3 are interspersed between the LSTMs and the subsequent dense layers. The final output is processed through a Softmax activation function to produce a probability distribution over possible next pitches.
The network uses Cross-Entropy as the loss function and RMSProp as the optimizer. Once trained, the model is "seeded" with a random sequence from the dataset. It predicts the next note, appends it to the sequence, drops the oldest note using a sliding window approach, and repeats the process to compose a full piece.
Data Processing
The study utilizes a dataset of 70 single-track piano MIDI files. The Music21 toolkit is used to extract pitches and chords, which are then converted into a stream of integers using a pitch-to-integer dictionary. These integers are transformed into one-hot encodings for network processing. The data is structured into groups of 81 pitches: the first 80 serve as the input sequence, and the 81st serves as the target label.
Results and Analysis
A core insight of this work is treating musical notes similarly to characters in a language model. This suggests that the structural "grammar" of music can be learned through statistical patterns without explicit hard-coded rules of music theory.
Training Performance
The model was trained for 43 hours over 72 epochs. Training was manually halted when the loss rate reached 0.4037, as the author observed that the loss rate began to increase after this point, indicating the onset of overfitting.
Quantitative Comparison
A comparative analysis of hidden layer sizes revealed a direct correlation between the number of neurons and the model's ability to abstract musical characteristics. As shown in the experimental results, a network with 1024 units significantly outperformed smaller configurations (128, 256, or 512 units), exhibiting a much steeper and lower loss curve.
Conclusion and Future Work
While the LSTM architecture effectively captures musical dependencies, several limitations remain. The high computational cost—nearly two days for a small dataset—suggests a need for architectural optimization. Furthermore, the current focus on single-track piano music limits the complexity of the generated output. Future work will explore polyphonic music and multi-track orchestrations, as well as the implementation of adaptive learning rate schedules like Adam to improve training efficiency.
Summary updated