Diversity-Authenticity Co-constrained Stylization for Federated Domain Generalization in Person Re-identification

Fengxiang Yang, Zhun Zhong, Zhiming Luo, Yifan He, Shaozi Li, Nicu Sebe

Introduction

Person Re-identification (re-ID) is a critical task in computer vision, focused on matching individuals across non-overlapping camera networks. While deep learning models have achieved significant success in controlled environments, they often struggle with domain shift—the performance drop encountered when a model is deployed in a new, unseen environment.

Standard approaches to this problem involve Domain Generalization (DG), which requires training on diverse datasets to learn universal, environment-agnostic features. However, in real-world scenarios, data is frequently "siloed" across different organizations due to strict privacy regulations. This necessitates Federated Domain Generalization (FedDG), where models must learn to generalize without ever centralizing sensitive image data.

Existing FedDG methods for re-ID face two primary hurdles: the inability to perform cross-domain stylization due to decentralization, and the "authenticity trap," where locally generated synthetic data becomes too unrealistic to be useful for training. To address these, researchers have introduced DACS (Diversity-Authenticity Co-constrained Stylization).

The DACS Framework

The core of DACS is a decentralized stylization approach that allows each local client to "hallucinate" new environments using only its own data. This is achieved through a Style Transformation Model (STM) that learns to shift and scale image statistics (channel-wise mean and variance) to simulate novel domains.

1. Diversity Constraint: Enlarging Wasserstein Distance

To ensure the STM doesn't simply replicate the local style, DACS employs a diversity constraint. By mathematically enlarging the Wasserstein distance between the original local data distribution and the transformed data distribution, the model is forced to generate styles that are as distinct as possible from the source.

Figure 4: The Diversity Constraint mechanism enlarging Wasserstein Distance to generate novel styles.

2. Authenticity Constraint: The Entropy "Sandwich"

Generating diverse data is only half the battle; the data must also remain authentic. If the STM generates "garbage" data that loses the person's identity, the model's performance will degrade. DACS introduces a novel authenticity constraint using an entropy-based relationship between two models:

  • Local Expert: A model trained exclusively on local data.
  • Global Model: The generalized model being trained across the federation.

The constraint enforces that the global model should find the stylized image harder to recognize than the original, but crucially, the local expert should find it even harder. This ensures the generated image represents a plausible "out-of-domain" style rather than just local noise or unrecognizable artifacts.

Experimental Setup and Results

The authors evaluated DACS using four major re-ID benchmarks: Market-1501 (M), CUHK02 (C2), CUHK03 (C3), and MSMT17 (MS). Following a standard leave-one-out protocol, the model was trained on three domains and tested on the fourth unseen domain.

Key Performance Metrics

DACS achieved state-of-the-art results across all scenarios. For instance, when testing on Market-1501 after training on the other three datasets, DACS reached 36.3% mAP, significantly outperforming existing federated methods like FedPav (25.4%) and FedReID (30.1%). Furthermore, the framework proved to be backbone-agnostic, showing strong performance with both ResNet-50 and Vision Transformer (ViT) architectures.

Key Insights and Ideas

The success of DACS highlights several important takeaways for federated learning:

  • Decentralized Stylization: It is possible to simulate global variety without ever seeing external data by intelligently pushing local statistics away from their mean.
  • Balancing Diversity and Authenticity: The "Entropy Sandwich" provides a robust way to ensure synthetic data remains within the bounds of realism while still being challenging enough to promote generalization.
  • Superior Regularization: In some cases, DACS outperformed centralized "Joint" training, suggesting that the synthetic diversity it generates acts as a more powerful regularizer than simply pooling a limited number of real-world datasets.
Figure 6: Overview of DACS balancing Diversity and Authenticity in feature and image space to achieve robust generalization.

Conclusion and Future Work

DACS provides a powerful solution for the privacy-constrained challenge of Federated Domain Generalization in re-ID. While the current implementation relies on channel-wise statistics for style transfer, future work may explore more complex generative models (like GANs) to provide even richer diversity. The authors also suggest that the diversity-authenticity balance could be a foundational principle for other federated tasks, such as medical imaging or autonomous driving, where domain shift is a critical concern.

Summary updated

Abstract

This paper tackles the problem of federated domain generalization in person re-identification (FedDG re-ID), aiming to learn a model generalizable to unseen domains with decentralized source domains. Previous methods mainly focus on preventing local overfitting. However, the direction of diversifying local data through stylization for model training is largely overlooked. This direction is popular in domain generalization but will encounter two issues under federated scenario: (1) Most stylization methods require the centralization of multiple domains to generate novel styles but this is not applicable under decentralized constraint. (2) The authenticity of generated data cannot be ensured especially given limited local data, which may impair the model optimization. To solve these two problems, we propose the Diversity-Authenticity Co-constrained Stylization (DACS), which can generate diverse and authentic data for learning robust local model. Specifically, we deploy a style transformation model on each domain to generate novel data with two constraints: (1) A diversity constraint is designed to increase data diversity, which enlarges the Wasserstein distance between the original and transformed data; (2) An authenticity constraint is proposed to ensure data authenticity, which enforces the transformed data to be easily/hardly recognized by the local-side global/local model. Extensive experiments demonstrate the effectiveness of the proposed DACS and show that DACS achieves state-of-the-art performance for FedDG re-ID.
BibTeX
@inproceedings{yang2024diversity,
  author = {Fengxiang Yang and Zhun Zhong and Zhiming Luo and Yifan He and Shaozi Li and Nicu Sebe},
  title = {{Diversity-Authenticity Co-constrained Stylization for Federated Domain Generalization in Person Re-identification}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2024},
  doi = {10.1609/aaai.v38i6.28468},
}
Figure