Traditional image and video compression—like JPEG or H.264—was designed for the Human Visual System (HVS). These codecs discard information that the human eye cannot perceive, such as high-frequency textures or subtle color shifts in dark areas. However, as deep learning models become the primary "consumers" of visual data, this human-centric approach creates a bottleneck.
Recent research suggests a shift toward Machine-centric Image Quality Assessment (MIQA) and task-specific compression. The core insight is that AI models do not "see" like humans; they can often perform perfectly on images that look "garbage" to us, yet fail on "visually lossless" images where critical semantic features have been smoothed away.
The Sensitivity Spectrum
Not all machine vision tasks are created equal when it comes to data degradation. Research across various domains shows a clear hierarchy of sensitivity:
- Global Semantic Tasks: Image classification and binary categorization are remarkably robust. Michael Kuchnik, George Amvrosiadis, and Virginia Smith demonstrate in Progressive Compressed Records that models can train on low-fidelity data with over 50% bandwidth reduction without losing final accuracy.
- Localized Detection Tasks: Object detection (bounding boxes) can tolerate significant compression—up to 10x in autonomous driving contexts—before performance drops, as noted by Madhumitha Sakthi and colleagues in their study of Fisheye Camera Visual Perception.
- Dense Perception Tasks: Semantic segmentation and depth estimation require pixel-perfect boundaries. Kristian Fischer and Markus Hofbauer highlight in Evaluation of Video Coding for Machines that these tasks are the first to fail as bitrates decrease because compression artifacts blur the very edges the models rely on.
The Storage Scaling Law
If you have a fixed storage budget (e.g., 1 Terabyte), should you store 10,000 high-quality images or 100,000 highly compressed ones?
Katherine L. Mentzer provides a definitive answer in Scaling Training Data with Lossy Image Compression. By deriving a "storage scaling law," Mentzer proves that it is almost always better to prioritize quantity over quality. Training on a larger volume of lossy data allows a model to see more diverse examples, which outweighs the "noise" introduced by compression artifacts.
This is further supported by João Maria Janeiro and Jakob Verbeek, who found that the performance drop from compression isn't usually due to "missing information," but rather a distribution shift. By simply finetuning models on compressed images, they recovered nearly all lost accuracy, even at 12x compression ratios.
Measuring What Machines Care About
Because traditional metrics like PSNR (Peak Signal-to-Noise Ratio) correlate poorly with model performance, researchers are building new evaluation frameworks. Yun Zhang and Xiaoqi Wang introduced the MIQD-2.5M dataset, shifting the paradigm to Machine-centric Image Quality Assessment. Their RA-MIQA model predicts how much a specific distortion will degrade a model's reliability, allowing for "content-adaptive" compression that preserves only the features a machine needs.
Progressive and Scientific Compression
In fields like connectomics, data volumes are so vast that traditional storage is impossible. Fuming Yang and Yaron Meirovitch developed a VQ-VAE with a Transformer prior that achieves 1000-fold compression for Electron Microscopy images.
Their "pay-as-you-decode" approach epitomizes the future of this field:
- Extreme Compression: For global browsing and coarse structural analysis.
- Selective Reconstruction: High-resolution details are only "unpacked" for fine-grained tasks like organelle texture analysis.
By treating compression as a learnable, task-aware component of the ML pipeline rather than a fixed preprocessing step, researchers are effectively "taking a byte" out of the data wall, enabling larger models and more efficient scientific discovery.