Q2D2: A Geometry-Aware Audio Codec Leveraging Two-Dimensional Quantization

Tal Shuster, Eliya Nachmani

Abstract

Q2D2 (Two-Dimensional Quantization) is a novel audio quantization scheme that groups latent features into pairs and maps them onto structured 2D grids, such as rhombic or hexagonal tilings. This approach allows the codec to capture inter-channel correlations more efficiently than 1D methods while avoiding the optimization instabilities and codebook underutilization common in high-dimensional vector quantization (VQ). Experimental results show that Q2D2 achieves state-of-the-art reconstruction quality at ultra-low bitrates (e.g., 1–3.3 kbps), outperforming existing models with significantly fewer tokens per second.

Introduction

In neural audio codecs, the primary challenge is compressing high-bitrate audio into a sequence of discrete tokens suitable for Large Language Models (LLMs). Current strategies typically fall into two categories: Vector Quantization (VQ), which is powerful but prone to "codebook collapse" and optimization instability, and Finite Scalar Quantization (FSQ), which is stable but ignores inter-channel correlations by quantizing each channel independently.

Q2D2 addresses this "dimensionality gap" by moving from 1D lines to 2D geometric planes. By quantizing pairs of channels together, the model captures local dependencies without the complexity of high-dimensional VQ.

Methodology

The Q2D2 framework integrates into standard encoder-decoder architectures (specifically building on WavTokenizer). The encoder output is projected into a $d$-dimensional space, bounded by a tanh nonlinearity to $[-1, 1]$, and then grouped into $P = d/2$ pairs. Each pair is mapped to the nearest point on a predefined 2D geometric grid.

Comparison of 2D grid tilings: (a) Hexagonal, (b) Rectangular, and (c) Rhombic.

The authors explored three tiling geometries as shown above. Unlike VQ, Q2D2 uses an implicit codebook defined by the product of grid levels, eliminating the need to store large embedding tables. Gradients are handled via a Straight-Through Estimator (STE).

Experimental Setup

The model was trained on extensive datasets, including ~150,000 hours of multilingual speech (Emilia, MLS) and ~8,000 hours of English speech (LibriTTS, VCTK). Evaluation spanned general audio benchmarks (AudioSet) and semantic richness tests (ARCH benchmark).

Results and Discussion

Geometric Efficiency

Mutual Information (MI) analysis confirmed that while features are independent prior to quantization, the 2D grid introduces structured dependencies that enhance compression density. The Rhombic grid emerged as the most effective geometry due to its superior packing efficiency and alignment with neural feature distributions.

Performance Benchmarks

  • Perceptual Quality: At 3.3 kbps (166 tokens/s), Q2D2 achieved a higher UTMOS score than the 9 kbps DAC model.
  • Ultra-Low Bitrate: At 1 kbps (75 tokens/s), Q2D2 outperformed FSQ and VQ variants of WavTokenizer across PESQ and STOI metrics.
  • Semantic Richness: In the ARCH benchmark, Q2D2 tokens captured more semantic information (e.g., speaker ID, emotion) at 53 tokens/s than competing models at much higher rates.

Conclusion

Q2D2 demonstrates that geometric awareness in quantization can bridge the gap between simple scalar methods and complex vector methods. By leveraging 2D grids, specifically rhombic tilings, the codec achieves high fidelity and semantic richness at extremely low bitrates. Future work includes extending this approach to 3D tilings and evaluating its performance in downstream generative tasks like text-to-speech.

Summary updated

BibTeX
@inproceedings{shuster2026q,
  author = {Tal Shuster and Eliya Nachmani},
  title = {{Q2D2: A Geometry-Aware Audio Codec Leveraging Two-Dimensional Quantization}},
  booktitle = {International Conference on Machine Learning},
  year = {2026},
  url = {https://openreview.net/forum?id=Ja9jo9pDvq},
}
Figure