Mass Personalization of Deep Learning

Johannes Schneider, Michail VlachosarXiv 2019 · OpenReview

Introduction

Standard machine learning models typically follow a "one-size-fits-all" paradigm, optimized for average performance across large populations. However, in domains such as personalized medicine, digital assistants, or handwriting recognition, individual data often deviates significantly from the global mean due to unique styles or preferences. This paper investigates methods to tailor deep learning models to specific individuals, addressing the dual challenges of data scarcity and the potential for "catastrophic forgetting" of general knowledge.

Methodology

The researchers propose two primary strategies for personalization: Shaping and Data Grouping.

Shaping Techniques

Shaping, or curriculum learning, adjusts the temporal order and frequency of individual data ($D_I$) relative to global data ($D_G$). Three specific approaches were evaluated:

  • Early Shaping (ES): Initial training on individual data followed by global data.
  • Late Shaping / Transfer Learning (TL): Pre-training on global data followed by fine-tuning on individual data.
  • Sample Weighing (SW): Concurrent training on both datasets with a higher sampling frequency for individual data.
Comparison of shaping techniques: Early Shaping, Sample Weighing, and Transfer Learning.

Data Grouping

To mitigate individual data scarcity, the authors augment personal datasets by identifying similar samples in the global pool using an Auto-encoder's latent space.

  • Single Sample Grouping (SG): Augments each user sample with the $n$ most similar global samples.
  • Individual Grouping (IG): Includes data from other users with similar overall distributions.

Experimental Setup

The study utilized the NIST Special Database 19, modified to include 30 classes (including flipped characters) to increase complexity. Experiments involved 100 users, each providing ~300 samples, against a global dataset of 30,000 samples. The models were evaluated on their ability to handle "hidden factors" (style variations) and "class counts variations."

Illustration of data variations: Hidden Factors (style) vs. Class Counts (subset of classes).

Key Results and Discussion

The evaluation, conducted using a VGG-style architecture, revealed several critical insights:

  • Performance Leaders: Sample Weighing achieved the highest personalization accuracy (98.4%), while Single Sample Grouping proved most effective among grouping strategies (97.9%).
  • The Early Shaping Surprise: Contrary to the standard practice of fine-tuning (Late Shaping), Early Shaping was found to improve both personal and global performance in some cases, highlighting the importance of initial feature shaping.
  • The Personalization Trade-off: A Pareto front exists between individual and global accuracy; maximizing one often necessitates a decrease in the other.
  • Similarity vs. Volume: High-precision augmentation with highly similar data is more effective than adding large volumes of less relevant data.

Conclusion

While mass personalization significantly boosts individual accuracy, it introduces challenges regarding computational scalability and data privacy. Future work will focus on these areas and further explore class variation scenarios. This research underscores that personalization is not merely a fine-tuning step but a complex optimization problem involving data selection and training schedules.

Summary updated

Abstract

We discuss training techniques, objectives and metrics toward personalization of deep learning models. In machine learning, personalization addresses the goal of a trained model to target a particular individual by optimizing one or more performance metrics, while conforming to certain constraints. To personalize, we investigate three methods of ``curriculum learning`` and two approaches for data grouping, i.e., augmenting the data of an individual by adding similar data identified with an auto-encoder. We show that both ``curriculuum learning'' and ``personalized'' data augmentation lead to improved performance on data of an individual. Mostly, this comes at the cost of reduced performance on a more general, broader dataset.
BibTeX
@misc{schneider2019mass,
  author = {Johannes Schneider and Michail Vlachos},
  title = {{Mass Personalization of Deep Learning}},
  year = {2019},
  eprint = {1909.02803},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1909.02803},
}
Figure