Data privacy protection in microscopic image analysis for material data mining

Boyuan Ma, Xiang Yin, Xiaojuan Ban, Haiyou Huang, Neng Zhang, Hao Wang, Weihua XuearXiv 2021 · arXiv

Abstract

Collaborative training of deep learning models for material microstructure analysis is often hindered by "data islands" and privacy concerns. This paper introduces FedTransfer, a federated learning framework that leverages Generative Adversarial Networks (GANs) to share image "style" rather than sensitive raw data. By decoupling material structure from imaging conditions, FedTransfer enables robust model training across heterogeneous datasets while preserving intellectual property. Experimental results on austenite and polycrystalline iron datasets demonstrate that FedTransfer significantly outperforms standard federated learning and approaches the performance of centralized training.

Introduction: The "Data Island" Challenge

In materials informatics, high-quality microscopic images are essential for training deep learning models to perform tasks like grain boundary identification. However, these datasets are often small and geographically dispersed across different research institutions. This creates "data islands" where labs possess valuable but isolated data.

Sharing raw experimental images is frequently prohibited due to intellectual property concerns. Furthermore, models trained on one lab's data often generalize poorly to others because of "Non-IID" (non-independent and identically distributed) issues—variations in microscope settings and sample preparation create distinct visual "styles" that confuse standard models.

Methodology: FedTransfer

The researchers propose FedTransfer, a framework that bridges these data islands by combining Federated Learning (FL) with Image Style Transfer.

Decoupling Style and Structure

The core innovation of FedTransfer is the realization that a microstructure image can be decomposed into two components: Structure (the proprietary geometry and arrangement of the material) and Style (the insensitive lighting, noise, and texture characteristics of the imaging process).

Decoupling material microstructure into ground truth labels (structure) and style information.

By isolating the "style," labs can share information about their imaging environment without revealing the sensitive "structure" of their materials.

The FedTransfer Workflow

FedTransfer operates through a multi-step process:

  1. Local GAN Training: Each client (lab) trains a GAN (specifically pix2pixHD) to learn its unique image style based on its local data.
  2. Style Model Exchange: Instead of sharing raw data, labs exchange these trained style models via a central server.
  3. Synthetic Data Generation: Lab A uses Lab B’s style model to transform its own private labels into synthetic images that mimic Lab B’s visual characteristics.
  4. Federated Training: Each lab now trains on a diverse dataset comprising its own real images and synthetic "foreign-style" images. The resulting model weights are then aggregated using the standard Federated Averaging (FedAvg) algorithm.
The FedTransfer workflow utilizing shared GAN models for privacy-preserving data augmentation.

Experimental Evaluation

The framework was tested using two distinct datasets: Austenite (optical images) and Polycrystalline Iron (serial section optical images). To simulate a real-world scenario, these datasets were assigned to different clients to create a Non-IID environment.

Key Results

FedTransfer was compared against three benchmarks: local training only, standard FedAvg, and centralized training (where all data is pooled).

  • Superior Generalization: FedTransfer achieved a Mean Average Precision (MAP) of 0.575 on a global test set, significantly outperforming standard FedAvg (0.510).
  • Closing the Gap: The performance of FedTransfer was remarkably close to the "gold standard" of centralized training (MAP 0.590), despite never sharing raw data.
  • Consistency: The method showed improvements in other metrics like the Adjusted Rand Index (ARI) and Mean Variation of Information (MVI), indicating more accurate and stable image segmentation.

Conclusion and Future Directions

FedTransfer effectively addresses the privacy and generalization challenges in materials science AI. By sharing "styles" rather than "structures," it allows institutions to collaborate without compromising intellectual property.

Future work will focus on reducing the communication overhead associated with transferring large GAN models through techniques like model quantization. Additionally, the researchers aim to scale the framework to include a larger number of diverse participants to further prove its robustness in complex, multi-institutional environments.

Summary updated

BibTeX
@misc{ma2021data,
  author = {Boyuan Ma and Xiang Yin and Xiaojuan Ban and Haiyou Huang and Neng Zhang and Hao Wang and Weihua Xue},
  title = {{Data privacy protection in microscopic image analysis for material data mining}},
  year = {2021},
  eprint = {2111.07892},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2111.07892},
}
Figure