DualComp: End-to-End Learning of a Unified Dual-Modality Lossless Compressor

Yan Zhao, Zhengxue Cheng, Junxuan Zhang, Qunshan Gu, Qi Wang, Li Song

Abstract

Most learning-based lossless compressors are designed for a single modality, requiring separate models for multi-modal data and lacking flexibility. However, different modalities vary significantly in format and statistical properties, making it ineffective to use compressors that lack modality-specific adaptations. While multi-modal large language models (MLLMs) offer a potential solution for modality-unified compression, their excessive complexity hinders practical deployment. To address these challenges, we focus on the two most common modalities, image and text, and propose DualComp, the first unified and lightweight learning-based dual-modality lossless compressor. Built on a lightweight backbone, DualComp incorporates three key structural enhancements to handle modality heterogeneity: modality-unified tokenization, modality-switching contextual learning, and modality-routing mixture-of-experts. A reparameterization training strategy is also used to boost compression performance. DualComp integrates both modality-specific and shared parameters for efficient parameter utilization, enabling near real-time inference (200KB/s) on desktop CPUs. With much fewer parameters, DualComp achieves compression performance on par with the SOTA LLM-based methods for both text and image datasets. Its simplified single-modality variant surpasses the previous best image compressor on the Kodak dataset by about 9% using just 1.2% of the model size.

Introduction

Lossless compression is transitioning from classical algorithms like Gzip or PNG toward learning-based models that leverage neural networks to predict data probabilities. However, current state-of-the-art (SOTA) methods face significant hurdles. High-performing models are often billion-parameter Large Language Models (LLMs) that, while effective, are extremely slow—sometimes requiring 30 minutes to decode a single 1080p image. Furthermore, most compressors are modality-specific; an image compressor cannot process text, and vice versa. While Multi-modal LLMs (MLLMs) could theoretically bridge this gap, their complexity makes them impractical for edge deployment. DualComp addresses these issues by providing a unified, lightweight, and efficient dual-modality lossless compressor.

Methods

DualComp is built on the RWKV-7 architecture, a lightweight alternative to Transformers that scales efficiently for long sequences. To handle the heterogeneity of image and text data, the authors introduced three structural innovations:

  1. Modality-Unified Tokenization: A shared 16K vocabulary handles both modalities. Text uses sub-word tokenization, while images are processed as sequences of sub-pixels from 16x16 patches.
  2. Modality-Switching Contextual Learning: To account for the different statistical properties of 1D text and 2D spatial images, DualComp employs separate projection layers for each modality, switching dynamically based on input.
  3. Modality-Routing Mixture-of-Experts (MoE): A router directs data to specialized "expert" sub-networks, allowing the model to develop modality-specific behaviors without a massive increase in parameters.
Architecture of DualComp showing modality-unified tokenization, switching contextual learning, and routing MoE.

Additionally, a reparameterization training strategy is used to enhance performance. High-rank matrix decomposition branches used during training are mathematically merged back into the main path for inference, maintaining a lean model size.

Experiments and Results

DualComp was trained on a balanced mix of ImageNet and enwik8. Evaluation was conducted on standard benchmarks: Kodak, CLIC, and DIV2K for images, and enwik9 for text.

Comparison of DualComp against SOTA single-modality and LLM-based compressors on image (Kodak) and text (enwik9) datasets.

Key results include:

  • Performance: On the Kodak dataset, DualComp-I (image-only) outperformed the previous SOTA by 9% while being only 1.2% of its size.
  • Efficiency: DualComp achieves inference speeds of 200 KB/s on desktop CPUs and 1.1 MB/s on mobile NPUs. It is nearly 90x faster than Llama3-8B for text compression.
  • Unified Capability: DualComp maintains high performance across both modalities, reaching 2.83 bits/Byte on Kodak and 1.10 bits/Byte on enwik9, matching the performance of much larger 8B-parameter models.

Discussion and Future Work

DualComp demonstrates that architectural efficiency and modality adaptation are more critical for compression than raw parameter scale. By sharing the majority of parameters while using small, modality-specific "switches," the model efficiently learns general data patterns alongside modality-specific structures. Analysis of the MoE layer revealed that images trigger more diverse expert routing than text, highlighting the statistical complexity of visual data.

While currently limited to image and text, the framework is designed to be extensible. Future work will explore adding audio and video support and investigating asymmetric parameter allocation to better handle modalities with varying compression difficulty.

Summary updated

Abstract

Most learning-based lossless compressors are designed for a single modality, requiring separate models for multi-modal data and lacking flexibility. However, different modalities vary significantly in format and statistical properties, making it ineffective to use compressors that lack modality-specific adaptations. While multi-modal large language models (MLLMs) offer a potential solution for modality-unified compression, their excessive complexity hinders practical deployment. To address these challenges, we focus on the two most common modalities, image and text, and propose DualComp, the first unified and lightweight learning-based dual-modality lossless compressor. Built on a lightweight backbone, DualComp incorporates three key structural enhancements to handle modality heterogeneity: modality-unified tokenization, modality-switching contextual learning, and modality-routing mixture-of-experts. A reparameterization training strategy is also used to boost compression performance. DualComp integrates both modality-specific and shared parameters for efficient parameter utilization, enabling near real-time inference (200KB/s) on desktop CPUs. With much fewer parameters, DualComp achieves compression performance on par with the SOTA LLM-based methods for both text and image datasets. Its simplified single-modality variant surpasses the previous best image compressor on the Kodak dataset by about 9% using just 1.2% of the model size.
BibTeX
@misc{zhao2025dualcomp,
  author = {Yan Zhao and Zhengxue Cheng and Junxuan Zhang and Qunshan Gu and Qi Wang and Li Song},
  title = {{DualComp: End-to-End Learning of a Unified Dual-Modality Lossless Compressor}},
  year = {2025},
  eprint = {2505.16256},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2505.16256},
}
Figure