Audio-Zero: Label-Free Self-Evolution for Fine-Grained Audio Reasoning

Siqian Tong, Xuan Li, Chaozhuo Li, Baolong Bi, Yiwei Wang, Yujun Cai, Shenghua Liu, Chengpeng HaoarXiv 2026 · arXiv

TL;DR

Audio-Zero is the first label-free self-evolution framework designed to improve fine-grained auditory perception and temporal reasoning in Large Audio Language Models (LALMs). By converting unlabeled audio contrast pairs into an "odd-one-out" self-play game, the model learns to generate highly detailed acoustic clues and reason over them without requiring any human annotations or ground-truth rationales.

What Problem Does This Paper Address?

While Large Audio Language Models (LALMs) excel at capturing broad semantic meanings (e.g., identifying that a clip contains a dog barking), they frequently stumble on fine-grained acoustic details, such as the precise order of events, repetition counts, or sound durations.

Fixing this usually requires massive, high-quality datasets with dense human annotations, which are incredibly expensive and difficult to scale. In text or vision domains, models can self-improve by verifying their outputs against discrete symbolic rules or static visual structures. However, audio is a continuous, temporally extended stream where crucial acoustic cues blend seamlessly into background noise. This makes direct verification via exact matching impossible, creating a bottleneck for label-free audio model training.

Methods

To bypass the need for human labels, Audio-Zero introduces an Auditory Self-Play Game optimized via Group Relative Policy Optimization (GRPO).

Figure 1: From external supervision to self-evolution. Existing audio-language learning paradigms depend on either external supervision or self-generated supervision. Audio-Zero instead converts unlabeled audio contrast pairs into a verifiable auditory self-play game, where supervision is derived from game outcomes, enabling self-evolution without external labels or reasoning traces.

The framework operates using unlabeled audio contrast pairs consisting of a Reference audio clip and a subtle Variant (e.g., the same sounds but in a different order). The training process is split into two main interactive stages:

  1. The Listening Stage (Clue Generation): The framework instantiates $N$ virtual players. $N-1$ players are given the Reference audio, while one "odd listener" receives the Variant. Over multiple rounds, the model—playing the role of each player sequentially—must generate natural-language clues describing what it hears without explicitly revealing its assigned role.
  2. The Attribution Stage (Odd-Listener Identification): The model shifts to a "judge" perspective. It reviews the public history of all generated clues alongside the Reference audio to deduce which player is the odd listener.
Figure 2: Overview of Audio-Zero. Audio-Zero formulates label-free auditory learning as a multi- round self-play game. Given an unlabeled audio contrast pair, players generate clues from different listening perspectives, while a judge identifies the odd listener. The known game outcome provides verifiable rewards that alternately optimize listening and attribution policies via GRPO, resulting in progressively improved auditory perception and reasoning.

Verifiable Rewards

Because the game environment procedurally assigns the odd listener, the true identity is known by construction. This allows the system to provide a verifiable reward without human answers:

  • Attribution Reward: A binary signal (1 or 0) based on whether the judge correctly identifies the odd listener.
  • Listening Reward: Combines a rule-based content utility score (rewarding specific acoustic terms and penalizing repetition) with a vote-aware penalty (penalizing clues that draw unwarranted suspicion or are trivially self-revealing).

The model alternates between optimizing the listening policy and the attribution policy, creating a competitive pressure that forces the emergence of highly detailed descriptions.

Datasets

  • Training: The authors sampled and filtered 2,000 unlabeled audio contrast pairs from Audio-Alpaca, a paired audio preference dataset.
  • Evaluation: The model was evaluated across three major benchmarks:
  • TREA: Targets fine-grained temporal reasoning across three subsets: Order, Count, and Duration.
  • MMAU Test-mini: A 1,000-question benchmark evaluating general audio understanding across Sound, Music, and Speech.
  • MMAR: A benchmark designed for deep reasoning over complex and mixed-modality audio.

Key Insights and Ideas

  • The Co-Evolution Loop: Alternating optimization creates a powerful feedback loop. As the attribution judge gets smarter, the players are forced to generate more discriminative, precise clues to win. Conversely, higher-quality clues make it easier to train a better judge.
  • Strategic Description Emergence: Thanks to the vote-aware penalty, the model doesn't just output longer text or blurt out answers. It learns to strategically balance informativeness with game effectiveness, leading to the natural emergence of fine-grained acoustic vocabulary (e.g., tracking exact durations and subtle background noises).
  • Preserving General Capabilities: Evolving on fine-grained temporal tasks did not degrade the model's performance on general audio tasks. Instead, the enhanced perceptual capabilities generalized broadly, improving scores across sound, music, and speech categories.

Key Results

The authors applied Audio-Zero to two strong open-source backbones: Qwen2-Audio-7B-Instruct and Qwen2.5-Omni-7B.

  • Outperforming Baselines: Audio-Zero consistently beat other label-free methods (like AQA-TTRL and Audio-CoT) and even outperformed strong label-dependent reinforcement learning baselines that had access to ground-truth answers.
  • Downstream Gains: On the temporal-heavy TREA benchmark, Audio-Zero improved Qwen2-Audio by 7.33% and Qwen2.5-Omni by 4.00% absolute accuracy.
  • Deep Reasoning Improvements: On the MMAR benchmark, Audio-Zero achieved substantial absolute accuracy boosts of 7.90% (Qwen2-Audio) and 10.00% (Qwen2.5-Omni) over the base policies.
Analysis of self-evolution dynamics showing increasing win rates, stabilizing clue lengths, and rising accuracy across downstream benchmarks over training iterations.

Limitations and Future Work

While Audio-Zero demonstrates the power of label-free evolution, it currently relies on pre-existing paired audio contrasts (from Audio-Alpaca) to seed the game. A compelling direction for future work is to automate the generation of these contrast pairs directly from raw, unpaired audio streams. Additionally, expanding the framework to support a larger number of players or more complex multi-agent conversational dynamics could further push the boundaries of LALM reasoning capabilities.

Summary updated

Abstract

Large Audio Language models (LALMs) have made rapid progress on acoustic understanding, yet they still struggle with fine-grained audio reasoning (e.g., recognizing event order, repetitions and duration). Existing post-training methods heavily rely on expensive external labels or provide only coarse semantic signals. To bridge this gap, we introduce Audio-Zero, the first label-free self-evolution framework in the field of LALMs that improves fine-grained auditory perception and reasoning. Audio-Zero constructs an auditory self-play game from unlabeled audio contrast pairs: most players hear a reference audio, while one odd listener hears a subtle variant. The model first generates clues describing what it hears and then identifies the odd listener by reasoning over inconsistencies among clues. Since the odd listener is known by construction, the game provides verifiable rewards without any annotated answers. Experiments with Qwen2-Audio-7B-Instruct and Qwen2.5-Omni-7B on TREA, MMAU Test-mini and MMAR show that Audio-Zero improves fine-grained audio reasoning while preserving broad audio understanding. Evolutionary and diagnostic analyses further reveal that increasingly fine-grained auditory descriptions emerge naturally from game pressure.
BibTeX
@misc{tong2026audio,
  author = {Siqian Tong and Xuan Li and Chaozhuo Li and Baolong Bi and Yiwei Wang and Yujun Cai and Shenghua Liu and Chengpeng Hao},
  title = {{Audio-Zero: Label-Free Self-Evolution for Fine-Grained Audio Reasoning}},
  year = {2026},
  eprint = {2607.20166},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2607.20166},
}
Figure