Reducing Distributional Uncertainty by Mutual Information Maximisation and Transferable Feature Learning

Jian Gao, Yang Hua, Guosheng Hu, Chi Wang, Neil Martin Robertson

Introduction

Deep learning models often struggle when deployed in environments that differ from their training data, a phenomenon known as domain discrepancy or distributional uncertainty. While humans easily recognize a car whether it is a photograph, a sketch, or a painting, Convolutional Neural Networks (CNNs) tend to over-rely on local textures (e.g., fur, metallic sheen) rather than global structural features (shape). This "texture bias" leads to poor generalization across domains.

Methodology: The MIMTFL Framework

The authors propose MIMTFL (Mutual Information Maximisation and Transferable Feature Learning) to address these limitations through two core components:

1. Quantifying Uncertainty via Mutual Information

The framework uses Mutual Information (MI) to rigorously quantify domain discrepancy. By maximizing Inter-domain MI, the model is incentivized to extract features shared between source and target domains. Conversely, Intra-domain MI is monitored to ensure consistency within a single domain, helping the model distinguish between useful signals and domain-specific noise.

2. MI-Guided Shape Learning

To force the network to prioritize structural information over transient textures, a self-supervised shape-learning task is introduced. Using standard edge detectors, the system generates "sketches" of training images. A dedicated shape-learning sub-network then attempts to reconstruct these edge maps from the backbone's internal features. Crucially, the weight of this task is dynamically adjusted based on the MI-calculated uncertainty: higher uncertainty triggers a stronger focus on shape-based features.

Key Insights: Shape as a Domain-Invariant Anchor

The core philosophy of MIMTFL is that while artistic style and pixel-level textures vary wildly across domains, the underlying geometry of an object remains relatively stable.

Figure 6: Various representations of the Eiffel Tower across domains (Sketch, Clipart, Infograph, Painting, Real), illustrating how structural shape remains consistent despite drastic changes in style and texture.

As shown in the figure above, the "skeleton" of the Eiffel Tower is recognizable across sketches, paintings, and photographs. By explicitly training for this structural consistency, the model learns to ignore the "noise" of artistic style.

Experimental Results

The framework was evaluated on the Office-Home and DomainNet benchmarks. DomainNet, featuring 345 categories across six diverse domains, represents one of the most challenging tests for domain adaptation.

  • Performance Gains: MIMTFL consistently outperformed state-of-the-art methods like CDAN and JAN. On difficult transfers (e.g., Sketch to Real), it improved accuracy by up to 77% over source-only baselines.
  • Domain Generalization: The model demonstrated robust performance even on completely unseen target domains, outperforming specialized algorithms like JiGen.

Conclusion

MIMTFL provides a mathematically grounded approach to reducing distributional uncertainty. By using Mutual Information as a "thermometer" for task difficulty and leveraging self-supervised shape learning as a structural anchor, the framework effectively decouples object identity from domain-specific style, leading to significantly more robust and transferable AI models.

Summary updated

Abstract

Distributional uncertainty exists broadly in many real-world applications, one of which in the form of domain discrepancy. Yet in the existing literature, the mathematical definition of it is missing. In this paper, we propose to formulate the distributional uncertainty both between the source(s) and target domain(s) and within each domain using mutual information. Further, to reduce distributional uncertainty (e.g. domain discrepancy), we (1) maximise the mutual information between source and target domains and (2) propose a transferable feature learning scheme, balancing two complementary and discriminative feature learning processes (general texture learning and self-supervised transferable shape learning) according to the uncertainty. We conduct extensive experiments on both domain adaption and domain generalisation using challenging common benchmarks: Office-Home and DomainNet. Results show the great effectiveness of the proposed method and its superiority over the state-of-the-art methods.
BibTeX
@inproceedings{gao2020reducing,
  author = {Jian Gao and Yang Hua and Guosheng Hu and Chi Wang and Neil Martin Robertson},
  title = {{Reducing Distributional Uncertainty by Mutual Information Maximisation and Transferable Feature Learning}},
  booktitle = {European Conference on Computer Vision},
  year = {2020},
  doi = {10.1007/978-3-030-58592-1\_35},
}
Figure