RandomMix: A mixed sample data augmentation method with multiple mixed modes

Xiaoliang Liu, Furao Shen, Jian Zhao, Changhai Nie

TL;DR

This paper introduces RandoMix, a novel data augmentation technique that enhances model robustness and diversity by intelligently combining linear and mask-based mixing strategies. RandoMix achieves state-of-the-art performance across various datasets without requiring additional computational overhead, making it a practical and effective addition to deep learning pipelines.

What Problem Does This Paper Address?

Deep neural networks, while powerful, are prone to overfitting, especially when training data is limited. Data augmentation is a common solution, but traditional methods often focus on variations within a single class. More recent "mixed-sample" augmentation techniques, like Mixup and CutMix, blend samples from different classes to improve generalization. However, existing methods often rely on complex saliency analysis for optimal mixing, which adds significant computational cost. Furthermore, many methods excel at either improving robustness (handling noise, occlusion) or diversity, but not both simultaneously. This paper aims to bridge this gap by proposing a method that enhances both robustness and diversity without incurring extra computational burden.

Methods

RandoMix is a mixed-sample data augmentation method that combines multiple mixing strategies. Its core novelty lies in its flexible approach:

  • Random Sample Pairing: It begins by randomly pairing samples within a mini-batch.
  • Random Mixing Method Selection: Instead of using a single mixing strategy, RandoMix randomly selects a mixing method from a predefined list of candidates (e.g., Mixup, CutMix, ResizeMix, Fmix). This selection can be weighted, allowing for control over the preference for certain mixing types.
  • Combined Mixing Modes: RandoMix leverages both "linear mixed modes" (like Mixup) and "mask mixed modes" (like CutMix). This combination is key to achieving both noise robustness and occlusion robustness.
  • Adaptive Mixing Ratio: The mixing ratio ($\lambda$) is sampled from appropriate distributions (Beta for linear, Uniform for others) based on the chosen mixing method.

This multi-modal, random selection approach allows RandoMix to adapt and improve performance across various challenges.

Datasets

The proposed RandoMix method was evaluated on several standard datasets:

  • CIFAR-10/100: Smaller image classification datasets with 10 and 100 classes, respectively.
  • Tiny-ImageNet: A smaller version of ImageNet with 200 classes.
  • ImageNet: A large-scale image classification benchmark with 1000 classes.
  • Google Speech Commands: A speech recognition dataset for recognizing spoken words.

Key Insights and Ideas

  • Synergy of Linear and Mask Mixing: The paper demonstrates that combining linear and mask-based mixing strategies is crucial for achieving comprehensive robustness against various challenges like adversarial noise, natural noise, and sample occlusion.
  • Diversity as a Key to Robustness: By increasing the diversity of mixed samples through random selection of mixing methods, RandoMix inherently improves model robustness.
  • No Additional Computational Cost: A significant advantage of RandoMix is that it integrates seamlessly into existing training pipelines and does not require extra computational overhead compared to baseline methods, unlike some saliency-based approaches.
  • Flexibility through Hyperparameters: The ability to adjust the candidate mixing methods and their weights offers a flexible way to fine-tune RandoMix for specific robustness or diversity requirements.

Key Results

RandoMix consistently outperformed existing state-of-the-art mixed-sample data augmentation methods across all tested datasets.

  • CIFAR-10/100: RandoMix achieved higher accuracy than Mixup, CutMix, and PuzzleMix, with notable improvements of up to 4.59% on CIFAR-100.
  • Tiny-ImageNet & ImageNet: RandoMix showed superior performance, achieving the highest Top-1 accuracy on Tiny-ImageNet and competitive results on ImageNet, all with comparable or lower computational cost than baselines.
  • Google Speech Commands: RandoMix achieved the most promising results, outperforming other methods and demonstrating its versatility beyond image tasks.
  • Robustness: On CIFAR-100, RandoMix exhibited the lowest "Clean Error" and significantly improved robustness against adversarial noise, natural noise, and sample occlusion compared to single-mode methods.

Limitations and Future Work

  • Hyperparameter Tuning: While RandoMix offers flexibility through parameters like Candidates and Weights, finding the optimal combination might require experimentation. The paper notes that the relationship between Weights and robustness isn't strictly linear due to inherent randomness.
  • Future Research Directions: The authors suggest exploring the nuanced interactions between different mixed modes, candidate selection strategies, and hyperparameters for even more robust and diverse training. They also propose extending RandoMix to more complex domains like multi-modal tasks, natural language processing, and reinforcement learning.

Summary updated

Abstract

Data augmentation is a very practical technique that can be used to improve the generalization ability of neural networks and prevent overfitting. Recently, mixed sample data augmentation has received a lot of attention and achieved great success. In order to enhance the performance of mixed sample data augmentation, a series of recent works are devoted to obtaining and analyzing the salient regions of the image, and using the saliency area to guide the image mixing. However, obtaining the salient information of an image requires a lot of extra calculations. Different from improving performance through saliency analysis, our proposed method RandomMix mainly increases the diversity of the mixed sample to enhance the generalization ability and performance of neural networks. Moreover, RandomMix can improve the robustness of the model, does not require too much additional calculation, and is easy to insert into the training pipeline. Finally, experiments on the CIFAR-10/100, Tiny-ImageNet, ImageNet, and Google Speech Commands datasets demonstrate that RandomMix achieves better performance than other state-of-the-art mixed sample data augmentation methods.
BibTeX
@misc{liu2022randommix,
  author = {Xiaoliang Liu and Furao Shen and Jian Zhao and Changhai Nie},
  title = {{RandomMix: A mixed sample data augmentation method with multiple mixed modes}},
  year = {2022},
  eprint = {2205.08728},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2205.08728},
}
Figure