Abstract
In recent years, deep discrete representation learning (DRL) has achieved significant success across various domains. Most DRL frameworks (e.g., the widely used VQ-VAE and its variants) have primarily focused on generative settings, where the quality of a representation is implicitly gauged by the fidelity of its generation. In fact, the goodness of a discrete representation remain ambiguously defined across the literature. In this work, we adopt a practical approach that examines DRL from a task-driven perspective. We propose a unified framework that explores the usefulness of discrete features in relation to downstream tasks, with generation naturally viewed as one possible application. In this context, the properties of discrete representations as well as the way they benefit certain tasks are also relatively understudied. We therefore provide an additional theoretical analysis of the trade-off between representational capacity and sample complexity, shedding light on how discrete representation utilization impacts task performance. Finally, we demonstrate the flexibility and effectiveness of our framework across diverse applications.
Introduction
Deep learning typically relies on continuous-valued vectors to represent data. However, complex reasoning and planning often benefit from discrete symbols—concepts or categories drawn from a fixed codebook. While frameworks like Vector Quantized Variational Auto-Encoders (VQ-VAE) are popular, they are primarily optimized for generative fidelity. This research shifts the focus of discrete representation learning (DRL) from "making pretty pictures" to "solving specific tasks." By introducing a unified, task-driven framework, we demonstrate that discrete representations can outperform continuous ones, particularly in low-data scenarios like reinforcement learning and domain generalization.
Task-Driven Framework
The proposed framework moves away from reconstruction loss, instead optimizing the discrete representation directly for a task-specific loss, such as classification or reward maximization.
To make the discrete bottleneck compatible with standard gradient backpropagation, we transform the constrained optimization problem into an unconstrained one. This involves using Wasserstein distance as a regularizer to align continuous encoder outputs with discrete codebook entries.
Theoretical Analysis: The Accuracy-Complexity Trade-off
A key contribution of this work is the formal proof of the Accuracy-Complexity Trade-off. While a larger codebook ($M$) increases the model's representational capacity (potential accuracy), the number of samples ($N$) required for generalization scales with $M^2$. This suggests a "Goldilocks" zone for codebook size: too small, and the model suffers from high bias; too large, and it requires impractical amounts of data, leading to high variance and overfitting.
Experimental Evaluation
The framework was evaluated in two domains: Visual Gridworld (Reinforcement Learning) and standard Domain Generalization benchmarks (PACS, VLCS, OfficeHome).
Discrete Representations as Denoisers
In the Visual Gridworld environment, where an agent navigates based on noisy images, the discrete bottleneck acts as a natural filter. By mapping noisy observations to a finite set of abstract states, the model ignores irrelevant pixel noise that often degrades the performance of continuous models.
Sample Efficiency and Performance
Results indicate that discrete representations are significantly more sample-efficient than continuous ones in low-data regimes.
As shown in the figure above, in the 10x10 Gridworld, discrete models outperformed continuous models when training data was scarce (3k and 10k samples). The continuous model only matched performance as the sample size reached 30k.
Fine-grained Discrete Adversarial Neural Network (FDANN)
For domain generalization, we introduced FDANN. Unlike standard methods that align data by broad class labels, FDANN uses a larger codebook to capture sub-modalities (e.g., "husky" vs "poodle" within the "dog" class). This allows for finer-grained alignment across domains. FDANN achieved state-of-the-art results, reaching 72.1% accuracy on OfficeHome and 88.7% on PACS.
Conclusion and Future Work
This work establishes that task-driven discrete representations provide a robust alternative to continuous ones, especially when data is limited. The primary limitation is the computational cost of calculating Wasserstein distances for latent alignment. Future research will focus on more efficient approximations or alternative alignment methods to scale the framework for larger datasets.
Summary updated