Automatic judges of AI-generated music fail in a specific way. Asked to score a song on several separate qualities, such as whether it follows the user's prompt, whether its structure is clear, and whether it is musical at all, they mostly report one blurred "is this good?" impression. In the best published music reward model, the musicality score and the prompt-alignment score move together almost in lockstep, with a rank correlation of about 0.85 (CMI-RewardBench). A beautiful song that ignores the prompt therefore escapes penalty. The same blur explains why signal-level metrics diverge from human listening. A metric can measure acoustic cleanliness, but listeners reward structure, phrasing and idiom, and a model trained on overall ratings has no reason to separate the two.
We propose to borrow one concrete structural device from interdisciplinary education, the horizontal de-scaffolding sequence, and use it as a training schedule for multi-aspect music reward models. In this classroom design, one subject is "in focus" while the supporting subjects arrive with pre-built help. The help is then removed as each subject rotates into focus (Horizontal CTE Curriculum). In our version, the scoring head for one musical dimension is trained while the model is handed the ratings of the other dimensions and a set of objective acoustic measurements. Those hand-ins are the scaffold. The head can only lower its error by learning what is specific to its own dimension, beyond what the shared quality factor and the objective metrics already explain. Focus then rotates and scaffolds are withdrawn in stages. The claim is narrow and testable: this schedule produces per-dimension scores that separate on counterfactual pairs, where one quality is good and another bad. It should do so better than ordinary multi-head training, a decorrelation penalty, or critique-then-score models, and without losing overall preference accuracy.
Where the two directions actually meet
The education literature contributes structure, not evidence. Its outcome data come from cohorts of 18 to 20 students (Horizontal CTE Curriculum; AI Education Across the Curriculum). One study also reports that confidence gained in one subdomain did not transfer to neighbouring ones (Self-Efficacy in ML Practices), which is a warning that skills learned under integration can stay local. We therefore treat de-scaffolding as a hypothesis about sequencing that ML experiments must earn on their own. We do not treat it as validated pedagogy.
The music-evaluation literature contributes the problem. Several papers motivate the gap. SongEval shows that human aesthetic judgment of full songs separates from acoustic fidelity: Audiobox-Aesthetics reaches only 0.61 to 0.66 Pearson correlation, against 0.912 for a fine-tuned predictor. The jazz study (Jazz Melody Generation) shows a distribution metric preferring the model that experts rated lower (overlapping area 0.7536 vs 0.7490, expert ratings 2.52 vs 2.85 out of 5), because idiomatic chromatic enclosures look like statistical error. The FreeSound loop benchmark reports that distribution scores can miss metrical structure.
Two papers constrain the design. CMI-RewardBench documents the 0.85 head entanglement and reward hacking under reinforcement learning. MuseCritic shows that direct scalar regression collapses scores into a 3.5 to 4.0 band. Its remedy is to generate a verbal critique before scoring, which costs a full autoregressive pass from an 8-billion-parameter audio language model on every evaluation.
The closest prior use of "curriculum" in audio is SARI. It orders question-answering data from easy to hard for reinforcement learning. That ordering is by difficulty within a single objective. It does not rotate between objectives or supply and then remove information from the other dimensions, and it addresses audio question answering rather than aesthetic scoring.
Core Claim
The reward model has a shared music encoder, a frozen-then-tuned MERT-style backbone of the kind probed in MARBLE, and one small head per dimension. During the focus phase for dimension $k$, head $k$ predicts the human rating from three inputs: the audio embedding, the true human ratings of the other dimensions, and a vector of objective measurements. The objective measurements are beat regularity from a beat tracker, pitch stability from F0 contours, and spectral distortion where a reference exists.
Because the other ratings already carry the general quality factor, and the objective vector already carries what signal metrics capture, the remaining gradient points toward dimension-specific, perceptual-beyond-objective information. Rotation then moves focus to the next dimension. Withdrawal happens in two steps. First, the true scaffold ratings are replaced by the model's own predictions, so the model "internalizes" the scaffold. Second, scaffold inputs are randomly dropped with a rate that rises to 1.
At deployment the model needs only audio and the prompt. That makes it cheap enough for best-of-N reranking or reinforcement learning, unlike critique-generating judges.
Research Program
The work has three parts.
The first part builds the model and the schedule on SongEval's five human-rated dimensions and CMI-Pref's musicality and alignment pairs. We pretrain on the 110,000 pseudo-labelled CMI-Pref-Pseudo pairs, following CMI-RM's two-stage recipe, so that our only change is the schedule.
The second part is a diagnostic set of counterfactual pairs, built without new annotation where possible. For alignment versus musicality, we re-pair each CMI-Pref clip with a mismatched prompt; the musicality is unchanged and the alignment is destroyed. For structure versus fidelity, we shuffle sections of SongEval songs, which keeps acoustic texture and destroys form. For phrasing versus fidelity, we apply codec degradation (low-bitrate Opus, as in the ViSQOL corpora of ViSQOL v3), which lowers fidelity while preserving phrasing. Each manipulation predicts which head should move and which should stay still. A small expert re-rating of 200 pairs checks that the manipulations really behave that way for listeners.
The third part tests the divergence question directly. On each SongEval dimension, we regress human ratings on the objective vector and measure how much of the remaining, unexplained part of the human rating each model's head captures. A model that merely re-learns objective metrics will score high on raw correlation and low on this residual.
Baselines
The naive reference is the off-the-shelf Audiobox-Aesthetics predictor, which SongEval reports at 0.61 to 0.66 Pearson correlation. Alongside it we report the objective vector alone, fitted by linear regression to each dimension, which shows how far signal metrics get on their own.
The strongest existing method is CMI-RM (CMI-RewardBench). It reports 78.20% accuracy on CMI-Pref musicality, 73.43% pairwise accuracy on Music Arena, and a head correlation of about 0.85. No reported result was found on Lacuna for its CMI-Pref alignment accuracy, so we will reproduce that number ourselves. We retrain it with our backbone, data and tuning budget, so the only difference is the schedule. Beating it means two things together. First, Music Arena and CMI-Pref accuracy must stay within one point of the reproduced CMI-RM. Second, head correlation on held-out CMI-Pref must fall well below 0.85, and accuracy on the counterfactual pairs must rise clearly. A gain in counterfactual accuracy that costs preference accuracy counts as a trade-off, not a win.
For SongEval per-dimension prediction, the bar is the fine-tuned UTMOS predictor at 0.912 Pearson. For Music Arena, MuseCritic's 71.35% serves as the critique-based reference. We compare against it at matched inference cost, and also at its full cost.
Several ablations isolate the ingredient. These are scaffolds with no rotation, rotation with no scaffolds, a randomly ordered rotation, a permanent scaffold that keeps true or predicted ratings at inference, and scaffolds made of objective features only versus other-dimension ratings only. Every variant receives the same number of tuning trials, the same total training steps and the same seeds, with three seeds each.
Why not just…?
Why not just add a decorrelation penalty between heads? This is the most direct competitor, and we take it seriously. A penalty pushes head outputs apart regardless of whether the data says the dimensions differ. It can therefore lower the 0.85 correlation by adding noise instead of finding real dimension-specific signal. The counterfactual pairs settle this. A penalty that only decorrelates should not improve accuracy on mismatched-prompt or shuffled-structure pairs. If it matches our schedule on those pairs, we will report that the pedagogical framing added nothing beyond the penalty.
Why not just train separate models per dimension? Separate models still each see audio whose ratings share a general quality factor. Nothing stops each one from learning that factor, so their outputs may correlate just as strongly. This is cheap to test, and we include it at matched total compute. It is a real threat if SongEval's dimensions are genuinely low in shared variance.
Why not just generate a critique first, as MuseCritic does? Critiques widen the score range (MuseCritic). However, they cost an autoregressive pass per clip, and the file reports no evidence that they reduce entanglement between dimensions. We compare head correlation and counterfactual accuracy against a critique-then-score model trained on the same data. We also report throughput, because the case for our approach rests partly on use inside reinforcement-learning loops.
Why not just order the data from easy to hard, as SARI does? Difficulty ordering changes which examples arrive first, but every head is still trained on the same blurred target. The randomly ordered rotation ablation and a SARI-style difficulty schedule run as separate arms. If the difficulty schedule alone closes the gap, the mechanism is ordering, not scaffolding.
Decisive Checks
The proposal succeeds only if three things hold together, all against the retrained CMI-RM. Overall preference accuracy on CMI-Pref and Music Arena stays within a point. Musicality–alignment rank correlation falls substantially below 0.85. Counterfactual-pair accuracy, confirmed on the expert-rated subset, improves over the decorrelation-penalty and separate-model baselines.
A secondary check addresses reward hacking. We use each reward model to rerank best-of-16 outputs from an open song generator such as YuE or DiffRhythm, then have experts rate the selections. The test is whether the schedule's selections are less often "acoustically polished but ignoring the prompt." Failure in any of the three primary checks means the rotating-focus idea does not hold up for this problem. That failure would itself be a useful negative result for music reward models.
References
- CMI-RewardBench: Evaluating Music Reward Models with Compositional Multimodal Instruction
- Design and Development of a Horizontal CTE Curriculum to Prepare Students for the New Manufacturing Economy (Work in Progress)
- AI Education Across the Curriculum: Design and Pilot Study of a Cross-Disciplinary Module Set
- Examining High School Students’ Self-Efficacy in Machine Learning Practices
- SongEval: A Benchmark Dataset for Song Aesthetics Evaluation
- Improving Automatic Jazz Melody Generation by Transfer Learning Techniques
- A Benchmarking Initiative for Audio-domain Music Generation using the FreeSound Loop Dataset
- MuseCritic: Learning Multi-Aspect Song Rewards through Natural-Language Aesthetic Critiques
- SARI: Structured Audio Reasoning via Curriculum-Guided Reinforcement Learning
- MARBLE: Music Audio Representation Benchmark for Universal Evaluation
- ViSQOL v3: An Open Source Production Ready Objective Speech and Audio Metric