Abstract
Just Noticeable Difference (JND) estimation is a critical component for perceptual image processing, yet it is often limited by data scarcity and rigid cross-modality alignment. We present MetaJND, a novel meta-learning framework that dynamically aligns RGB data with saliency and depth information. By utilizing a Multi-Modal Meta Alignment (MMA) module, the model generates adaptive weights to handle unseen data, overcoming the limitations of static feature extraction. Experimental results demonstrate that MetaJND achieves a 24.11% average coding gain across standard codecs and superior performance in perceptual metrics compared to state-of-the-art methods.
Introduction: The Challenge of JND Modeling
Accurately modeling the Just Noticeable Difference (JND)—the maximum amount of distortion human vision can tolerate—is essential for perceptual image compression and watermarking. By identifying "invisible" data, systems can save significant bandwidth without degrading quality.
However, current data-driven JND models face two major hurdles:
- Data Scarcity: Subjective JND labeling is expensive and time-consuming, leading to small datasets and poor generalization.
- Modality Limitations: Existing models often fail to effectively align auxiliary modalities like depth or saliency with the primary RGB signal, or they rely on misleading semantic information.
The MetaJND Architecture
MetaJND addresses these issues by moving away from static feature extraction toward a dynamic, meta-learning-based approach.
The architecture consists of three primary components:
- Multi-Modal Meta Alignment (MMA): This module uses Swin Transformers and cross-attention to correlate RGB features with auxiliary modalities. A Meta-learning Module (MM) generates dynamic weights, allowing the network to adapt its feature space on the fly for unseen inputs.
- Multi-layer Channel and Spatial Feature Rectification (MCSFR): Operating across hierarchical layers, MCSFR filters noise and rectifies features in both channel and spatial dimensions to preserve both coarse structures and fine details.
- Multi-Scale Feature Fusion (MSFF): This module progressively aggregates rectified features from different scales to produce the final high-resolution JND map.
Key Insights: Meta-Learning and Feature Alignment
A central innovation of MetaJND is the use of meta-learning to solve the "small data" problem. By learning how to adjust weights based on input characteristics, the model handles the diversity of natural images far better than fixed-weight CNNs.
Furthermore, the authors argue against using semantic segmentation as an auxiliary modality, noting that JND is driven by low-level physical properties like texture and depth rather than high-level object categories.
The effectiveness of this alignment is visually confirmed through t-SNE plots. As shown above, the MMA module pulls features from disparate modalities—which initially form separate clusters—into a cohesive, overlapping feature space, proving the effectiveness of the meta-alignment strategy.
Experimental Results
MetaJND was benchmarked against eight state-of-the-art methods across four datasets (PWJND, MCL-JCI, KonJND-1k, and MDTJND).
- Superior Accuracy: It consistently outperformed competitors in perceptual metrics, including SSIM, LPIPS, and MS-SSIM.
- Compression Gains: When integrated into standard codecs (JPEG, HEVC, and VVC), MetaJND achieved an average coding gain of 24.11%.
- Robust Watermarking: In digital watermarking tests, the MetaJND-guided approach showed a lower Bit Error Rate (BER) under various attacks compared to existing deep watermarking models.
Conclusion and Future Work
MetaJND represents a significant advancement in perceptual modeling by leveraging meta-learning to overcome data scarcity. While the model demonstrates impressive results, future work could address its computational overhead and extend the framework to handle temporal redundancies in video JND (VJND).
Summary updated